Article Text
PDF
Abstract
Introduction In January 2019, the WHO reviewed evidence to develop global recommendations on self-care interventions for sexual and reproductive health and rights (SRHR). Identification of research gaps is part of the WHO guidelines development process, but reliable methods to do so are currently lacking with gender, equity and human rights (GER) infrequently prioritised.
Methods We expanded a prior framework based on Grading of Evidence, Assessment, Development and Evaluation (GRADE) to include GER. The revised framework is applied systematically during the formulation of research questions and comprises: (1) assessment of the GRADE strength and quality rating of recommendations; (2) mandatory inclusion of research questions identified from a global stakeholder survey; and (3) selection of the GER standards and principles most relevant to the question through discussion and consensus. For each question, we articulated: (1) the most appropriate and robust study design; (2) an alternative pragmatic design if the ideal design was not feasible; and (3) the methodological challenges facing researchers through identifying potential biases.
Results We identified 39 research questions, 7 overarching research approaches and 13 discrete feasible study designs. Availability and accessibility were most frequently identified as the GER standards and principles to consider when planning studies, followed by privacy and confidentiality. Selection and detection bias were the primary methodological challenges across mixed methods, quantitative and qualitative studies. A lack of generalisability potentially limits the use of study results with non-participation in research potentially highest in more vulnerable populations.
Conclusion A framework based on GRADE that includes stakeholders’ values and identification of core GER standards and principles provides a practical, systematic approach to identifying research questions from a WHO guideline. Clear guidance for future studies will contribute to an anticipated ‘living guidelines’ approach within WHO. Foregrounding GER as a separate component of the framework is innovative but further elaboration to operationalise appropriate indicators for SRHR self-care interventions is required.
- epidemiology
- study design
- public health
- infections
- diseases
- disorders
- injuries
This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.
Statistics from Altmetric.com
Key questions
What is already known?
Reliable methods to identify research gaps from WHO guidelines are lacking with gender, equity and human rights (GER) infrequently prioritised when formulating future research questions.
What are the new findings?
A prior Grading of Evidence, Assessment, Development and Evaluation (GRADE)-based framework was revised to include mandatory stakeholder input and selection of relevant GER standards and principles and successfully applied to a WHO guideline on self-care interventions for sexual and reproductive health and rights (SRHR).
39 research questions, 6 research approaches and 16 discrete study designs for SRHR self-care were identified and the key methodological issues outlined.
Availability and accessibility were the GER standards and principles most frequently identified as relevant.
What do the new findings imply?
Application of the GRADE-informed framework to future WHO guidelines development processes has potential to harmonise research question formulation and to ensure more consistent consideration of GER across the organisation.
The dashboard outlining future SRHR studies is expected to prove useful to researchers.
Introduction
The mandate of the WHO is to develop global clinical and public health guidance that informs country-level healthcare policies, guidelines, programmes and services. The systematic process of WHO guideline development includes the identification and documentation of research gaps.1 This is intended to focus research to inform and strengthen new and existing WHO recommendations. However, wide variation in the quality, robustness, presentation and dissemination of research priorities across sets of WHO guidelines has been noted.2 Currently within the WHO transformation process, systematic approaches to coherent research agenda formulation are being explored.
In 2018, the WHO Department of Reproductive Health and Research embarked on a process to: (1) develop evidence-based recommendations on self-care interventions for sexual and reproductive health and rights (SRHR) and (2) delineate research gaps in the field. A Guidelines Development Group (GDG) was established comprising experts and stakeholders active in SRHR and/or self-care interventions. The GDG was regionally and gender representative and included representation from youth, vulnerable populations, healthcare providers, policymakers, programme managers, researchers and civil society, as well as experts in human rights, gender equality and health economics.3
The following definition of self-care was adopted for the guidelines: self-care is the ability of individuals, families and communities to promote health, prevent disease, maintain health and to cope with illness and disability with or without the support of a healthcare provider.4 The scope of self-care as described in this definition includes health promotion, disease prevention and control, self-medication, providing care to dependent persons, seeking hospital/specialist care if necessary and rehabilitation including palliative care.
Members of the GDG attended a scoping meeting in March 2018 to prioritise and refine the self-care interventions to be evaluated in the guideline and to reflect on discussions arising from a prior WHO-led meeting on the ethical, legal, social accountability and human rights implications of self-care interventions.5 The GDG agreed that the systematic consideration of gender, equity and human rights (GER) in the provision of self-care interventions is key to ensuring better health for all. As defined by WHO, human rights-based approaches to health consider and address inequalities, power imbalances and discrimination, including those related to gender.6 Such approaches also aim to support better and more sustainable health and development outcomes and focus on capacity development, both of duty bearers to meet their obligations and of individuals to claim their rights.7 Attention to ways of empowering individuals and communities, particularly vulnerable populations, to understand and claim their rights requires special attention in many domains, including in research. Sridharan and colleagues8 argue that WHO needs to develop concrete actions towards mainstreaming GER. The GDG selected the fulfilment of the following GER standards and principles as key to measuring the impact of self-care interventions in future research studies: (1) the right to highest attainable standard of health (including availability, accessibility, acceptability and quality); (2) active and fully informed participation; (3) non-discrimination; (4) the right to seek, receive and impart information; (5) informed decision-making; (6) privacy and confidentiality; and (7) accountability.9
At a final GDG meeting in January 2019, the GDG reviewed evidence from five systematic reviews on self-care interventions in order to formulate new consensus-based self-care recommendations.3 The new recommendations covered the following topics: (1) self-injectable hormonal contraception, (2) over-the-counter oral contraception, (3) home-based ovulation predictor kits (OPKs), (4) self-sampling for human papilloma virus (HPV) and (5) self-collection of samples for sexually transmitted infections (STIs). Each recommendation was formulated in response to an a priori clinical or public health question regarding the effectiveness of the intervention when offered as an additional approach to current practice within the formal health sector. The questions were formulated using the Population, Interventions, Comparison and Outcomes (PICO) structure.1
To further inform the guidelines, the GDG also reviewed evidence from a global online survey of healthcare providers and users of healthcare services, which was hosted on the WHO website and shared via several listservs between July and October 2018.10 The survey included a range of questions regarding respondents’ values and preferences for self-care interventions for sexual and reproductive health. Three of these were interventions evaluated in the systematic reviews: (1) self-injectable hormonal contraception, (2) oral contraception (including over the counter), and (3) self-collection of samples for STIs (including HPV). The survey reached 294 (35.6%) healthcare providers and 531 (64.4%) users of healthcare services from 113 countries. There was diversity in WHO regional representation in responses from healthcare providers (Africa 30.0%, Europe 20.8%, Latin America and the Caribbean 20.8%, Asia 14.2%, Northern America 13.9% and Oceania 0.3%) and lay respondents (Europe 32.0%, Africa 23.0%, Asia 20.2%, Northern America 12.9%, Latin America and the Caribbean 10.5% and Oceania 1.3%).10
In this article, we present a systematic approach to the formulation of research questions to guide future studies related to the five topics of the new WHO recommendations on self-care interventions. We demonstrate how evidence from systematic reviews, results from a global survey and consideration of GER, together informed question formulation. Our primary aim is to demonstrate both the feasibility and utility of using a structured process combined with a GER lens to identify research gaps within the context of developing a WHO guideline. A secondary aim is to provide researchers with a dashboard of potential evidence-informed research questions and related feasible study designs—and associated methodological challenges and GER considerations—to ensure responsive future research in the rapidly evolving field of SRHR self-care.
Methods
We adopted a similar methodological approach to research formulation used in two previously published WHO guidelines.11 12 Prior to the GDG meeting, a systematic review, including meta-analysis where appropriate, was conducted for each of the five selected PICO questions.13–17 The overall certainty of evidence was rated as high, moderate, low or very low according to the Grading of Evidence, Assessment, Development and Evaluation (GRADE) approach.18 The certainty of evidence is dependent on the risk of bias, precision, consistency, directness of the results and other considerations such as publication bias. During the meeting, the GDG formulated a recommendation in response to the following GRADE domains: certainty of the evidence, balance of benefits and harms, resource use implications, user values and preferences, acceptability among healthcare providers and key stakeholders, feasibility, equity and human rights (See table 1). Recommendations were then further categorised by the GDG as strong or conditional. In general, strong recommendations are made when the quality of evidence is high and the benefits of an intervention clearly outweigh the harms, whereas conditional recommendations recognise that the quality of the evidence is low or that specific country contextual factors may determine the uptake of a recommendation.19
Key domains that require consideration when formulating WHO recommendations
We used the GRADE framework as a starting point as it allows determination of research gaps based on the strength of the recommendation and the certainty of the evidence. For example, identification of a conditional recommendation, or low or very low certainty evidence, regardless of the strength of the recommendation, is indicative of where further research is required (see figure 1).
Hierarchical decision making algorithm to formulate research questions based on the presence and strength of a WHO recommendation, combined with the source of the question. GDG, Guidelines Development Group; GER, gender, equity and human rights; PICO, Population, Interventions, Comparison and Outcomes.
Previously, we had expanded the GRADE framework and included an additional component specific to the values and preferences of the community of users and potential users the recommendations intend to serve.12 We applied this step to the current guidelines on self-care interventions and reviewed results from the WHO-commissioned global online survey of users of sexual and reproductive self-care interventions.3 In addition, we captured specific research gaps identified by GDG participants during dedicated sessions at both the scoping and guidelines development meetings. We then viewed each research question through a GER lens and through discussion and consensus between authors of this paper identified the GER standards and principles essential to consider and measure when planning future research.
Following the GDG meeting, for each a priori PICO question, and for additional research questions identified from the global survey and during the meeting, we tabulated the following:
The clinical, programmatic, values-based or GER research question.
The related recommendation formulated by the GDG (or a record that no recommendation was made) where applicable.
The strength of each recommendation where applicable.
The certainty of the evidence underpinning the recommendation where applicable.
The GER standards and principles most relevant to the question as discussed during the scoping and guideline development meetings and agreed on by authors of this paper.
From the above, a research gap was identified when one or more of the following conditions was met: (1) the recommendation was conditional; or (2) the certainty of evidence was low or very low (even in the presence of a strong recommendation); or (3) key GER principles were not considered or absent (even if the certainty of evidence was moderate or high for clinical or public health effectiveness outcomes); or (4) no recommendation was made and the GDG or survey had articulated a research gap during their deliberations. We also identified a gap for strong recommendations if the evidence only arose from well-resourced settings. For each identified research gap, the authors then tabulated the following:
The most robust study design to answer the research question(s), including identification of GER principles to consider.
Conceptualisation of a pragmatic alternative study design if the ideal design was not feasible.
Consideration of the methodological challenges of the alternative study design with the potential bias(es) identified.
Following tabulation, we categorised the primary focus of each research question according to the relevant GRADE domain as outlined in table 1.
Patient and public involvement
Participation of the public in developing research gaps was achieved in several ways: (1) through completion of the global survey, (2) members of several civil society associations contributed to the GDG meeting as participants on the GDG, (3) one of the authors is a patient representative and (4) all authors identify as individuals who engage or may engage with SRHR self-care interventions.
Results
Tables 2–6 provide an overview of research gaps in SRHR self-care, the most relevant GER for each question and the study design(s) considered most feasible and appropriate to answer each question. Thirty-nine research questions were formulated with seven overarching research approaches and 13 discrete feasible study designs developed:
Research gaps for self-administration of injectable contraception
Research gaps for self-collection of STI samples
Research gaps for over-the-counter (OTC) oral contraception
Research gaps for self-sampling for human papilloma virus (HPV) infection
Research gaps for home-based ovulation predictor kits (OPKs)
Survey approach (one prevalence survey; two interrupted time series; five cross-sectional studies; one household diary study).
Qualitative approach (seven key informant interviews; three focus groups).
Implementation research approach (four prospective mixed methods cohort studies; one demonstration project).
Comparative effectiveness research (three non-randomised controlled trials (non-RCTs); two RCTs).
Economic approach (four cost-effectiveness studies; four costing studies).
Surveillance (one sentinel surveillance study).
Standard patient study (1).
We identified selection bias and detection bias as the primary methodological challenges across mixed methods, quantitative and qualitative studies. Detection bias was driven by concerns around self-reported data that may be prone to social desirability bias when participants provide answers they consider the assessors expect to hear. Selection bias in non-randomised studies limits generalisability and occurs when research participants who agree to participate differ qualitatively from those who do not agree to participate. Non-participation may be highest in those vulnerable populations who fear discrimination or have privacy and confidentiality concerns. This would then limit the utility of the results obtained from those who do participate, including in relation to GER.
The most frequent GER standards and principles considered relevant were availability and accessibility, followed by privacy and confidentiality.
Discussion
We developed a structured approach to identification of research gaps and formulation of research questions and study designs during a WHO guidelines development process. Our approach builds on our previous work in this area and is rooted in the current GRADE framework used by WHO, further extending it to incorporate the user perspective and foregrounding GER throughout the process.
To the best of our knowledge, our approach is unique in providing practical systematised steps to research question formulation during WHO guideline development and elaboration of the design of future studies. Prior work in this area has been sparse and focused on identification and characterisation of research gaps arising from systematic reviews (not guidelines) and has not described the type of research that is required.20 Given that the aim of identification of research gaps is to reduce waste and increase research value,21 collation and possible registration of research gaps on a publicly accessible platform should be encouraged to realign future studies with the existing body of evidence. This would increase access to suggested methodologies and processes to advance research from resource-constrained settings where we found gaps in existing evidence around self-care interventions. Given that much of the evidence reviewed came from high-income countries, it will also be important for future research in these fields to develop capacity where needed and promote research appropriate to the local contexts in low-income and middle-income countries.
The application of a ‘living guidelines’ approach to WHO guideline development process will ensure greater responsiveness to new research findings.22 Current guidance for formulating and prioritising research gaps during WHO guideline development processes requires strengthening in the WHO Handbook of Guideline Development.1 Research gap identification is rarely prioritised during guideline development processes and the opportunity to shape the future research agenda is missed.2 Our practical approach presents an opportunity to WHO (and other guidelines developers) to better integrate research identification and elaboration into the guidelines decision-making space, in particular research that is reflective of need and priorities. Time and cost constraints may preclude extensive discussion of research gaps during a guidelines meeting, but at a minimum a dedicated agenda item and a working group tasked to develop the research gaps further following the meeting should be considered. We can envision automation of the decision-making steps in our approach using the hierarchical model outlined in figure 1. However, we would caution that such a process be viewed as a starting point to facilitate more in-depth discussions rather than a rigid template.
GER are recognised as key considerations in the SRHR field.23 24 SRHR self-care interventions present many opportunities to address common obstacles to delivering equitable, gender-responsive and rights-based healthcare. Provided users are fully informed, able to make autonomous choices and are able to link to a health system when required, high-quality self-care interventions can offer available, accessible and acceptable healthcare to those individuals and groups who may be less likely to access formal healthcare due to fear of discrimination or privacy and confidentiality concerns. Benefits may flow to healthcare providers too as users become more engaged in their healthcare, with task-shifting reducing heavy workloads and consumer self-sampling potentially reducing the risk of personal injury to providers when obtaining samples (eg, via venepuncture).
We consistently identified accessibility as an important GER principle to measure across questions for all five interventions. From a human rights perspective, accessibility encompasses physical accessibility, economic accessibility (affordability), non-discrimination in access and information accessibility.25 This illustrates that multiple indicators are often required to measure a single outcome domain. Within a rights-based approach to measuring health, health indicators can be used to measure different standards and principles relating to GER.26 For example, some health surveys collect data on informed choice in contraception uptake (measured by the proportion of users who were informed of potential side effects, what to do in the case of side effects and alternative contraceptive options). This highlights the human rights standard of acceptability as well as the principle of autonomy, expressed through free, full and informed decision making.27 Similarly, indicators commonly used in the fields of GER may be used to measure certain aspects of sexual and reproductive health; for example, measures of the existence, content and degree of implementation of relevant laws and policies governing access to sexual and reproductive health information and services can help elucidate patterns in uptake of services, including for self-care initiatives. An indicator that incorporates both sets of concern will allow for identification of interventions that are sensitive to issues relating to GER and most effective in terms of improving health.26
In our approach, we identified the GER standards and principles most relevant to each question but did not operationalise measurement. This is an important next step. Development of, and agreement on, standardised indicators and the optimal instruments to measure these within the SRHR self-care intervention field will greatly enhance the usability and uptake of GER-informed evidence into future guidelines through streamlining meta-analysis and synthesis more broadly. The experience and guidance of the Core Outcome Measures in Effectiveness Trials (COMET) initiative are instructive in this regard. COMET advocates for the development and application of agreed, standardised sets of outcomes, known as ‘core outcome sets’, which represent the minimum that should be measured and reported in all clinical trials of a specific condition and are also suitable for use in clinical auditing or research other than randomised trials.28 Inclusion of GER in an SRHR self-care core outcome set will ensure GER are integral to all future evaluations. We welcome initiatives to bring researchers from the human rights, gender, health economics and epidemiology fields together to advance this.
The WHO aims to be representative when determining GDG composition, and for the guideline on self-care interventions, we believe reasonable representation was achieved. However, in general, the balance is in favour of healthcare providers rather than community members and users of interventions. We were also able to ensure participation of users and healthcare providers of SRHR self-care interventions by incorporating the findings of the global survey and focus group discussions with vulnerable populations into the formulation of research gaps. Active and informed participation of users is also an important consideration during implementation of recommendations and in planning future research studies. Many of the proposed future studies are focused on vulnerable populations, and participation in the study or assessment of an outcome may place participants at risk. This illustrates that outcome measurement and the process of research itself requires careful consideration of GER to minimise unintended harms. Community-based participatory research is a useful strategy to conducting research that is relevant, appropriate and acceptable.
Prior to conducting studies on self-care, researchers need to consider the burdens faced by study participants, particularly for implementation research and qualitative methodologies. There is an opportunity cost to participants who may have to take time away from work or family care in order to contribute to research activities. We encourage meaningful involvement of individuals or groups representing target populations in the design and cocreation of research methodologies to optimise benefits to individuals and the wider group while minimising burdens to the participants. This approach is supported by ethical considerations29 and several frameworks that promote person-centred interventions.30 31
Each of the proposed study designs can be further expanded into a template for a study protocol and made publicly available. The WHO successfully achieved rapid deployment of ethically approved clinical trial protocols for the Ebola vaccine under emergency conditions.32 33 Under more controlled conditions, WHO may consider systematically developing field-specific study protocol templates that meet ethics standards. Researchers can then modify these templates to their context addressing any specific local ethics requirements before rapidly conducting studies during implementation of new WHO recommendations to inform programme scale-up or for testing new strategies well in advance of planned future GDG meetings.
Similar to our previous work where we applied the original framework to SRHR guidelines for women living with HIV, we found that many of the identified research questions require evaluation of complex, multifaceted and often multisectoral interventions that may be best suited to evaluation within an implementation research paradigm.12 Implementation science provides a platform to learn whether an intervention works in real-world settings and demonstrates how to ensure an intervention can effectively be brought to scale. Importantly, outcomes such as acceptability, feasibility and costs are encouraged in addition to conventional measures of effectiveness.34 RCTs are the optimal design for providing evidence of efficacy and will remain the gold standard for informing WHO recommendations. However, comparative effectiveness research (CER) where available interventions are compared with each other (instead of with usual care or placebo) can be viewed as a bridge between RCTs and implementation research.35 36 Five of our research questions are best evaluated using CER (eg, comparison on OPKs with other fertility management options) with 10 studies requiring a multistudy approach best articulated in implementation science. The challenge for WHO will be how these types of study designs and approaches can best be incorporated into future WHO guidelines decision making. Some progress has been made with the inclusion of the Risk of Bias in Non-randomised studies of interventions (ROBINS-I) tool for assessing risk of bias in non-randomised studies in GRADE,36 guidance for rating certainty of evidence when reviewing public health interventions37 and the development of the WHO-INTEGRATE framework to WHO guidelines development that includes specific methods to incorporate norms and values and a greater complexity perspective.38
Our approach is unique to the development of normative guidance. As such it is limited by the scope of a guideline, which may be intentionally narrow due to feasibility, resources and time constraints. The research gaps arising from such a process are specific to the scope of the guidelines and should not be viewed as a comprehensive research agenda for the relevant clinical or public health field nor as a research prioritisation exercise. However, the resultant research questions can provide a baseline list of questions that can then be further subjected to one of several research prioritisation methodologies.39–41
Lastly, while our overall decision-making process is structured and systematised, the selection of the most relevant GER standards and principles was done primarily through iterative discussion and ultimately consensus between study authors. Such decisions may not readily lend themselves to standardised processes, but we would advocate that in order to ensure the integrity and generalisability of the selected standards and principles, decisions should be made by a diverse and representative group as possible. We attempted to do so by reflecting a diversity of backgrounds, skills and experiences among the study authors, but we acknowledge that our selections are nonetheless subjective and may be best done by the broader GDG. Further exploration and development of ways to standardise such decisions such as initiated in the family planning field is now required.42
Conclusion
A framework based on GRADE that includes stakeholders’ values and identification of core GER standards and principles provides a practical, systematic approach to identifying research questions from a WHO guideline. Uptake of this framework has the potential to harmonise methods and ensure more consistent consideration of research question formulation across the organisation. Clear guidance for future studies, including anticipation of, and methods to reduce risks of bias, can contribute to an anticipated ‘living guidelines’ approach within WHO. Foregrounding GER as a separate component of the framework is key to ensuring it is considered as integral to outcome evaluation, and further elaboration to operationalise appropriate indicators for SRHR self-care interventions is required.
Acknowledgments
The authors are grateful to all members of the Guidelines Development Group for their contributions during the meeting. We would like to thank Rachel Beanland, Independent Public Health Consultant, for her helpful comments on the selection of study designs and elaboration of biases.
Footnotes
Handling editor Seye Abimbola
Contributors NS conceived the concept and developed the framework and methodology informed by prior work in the area. MN led the WHO guideline development process and initiated the use of the framework during the guideline process. All authors contributed to refining the framework and provided expertise-informed contributions to adapting the framework, formulating research questions and identifying study designs. All authors contributed to the final manuscript.
Funding This paper was funded by WHO/UNDP-UNFPA-UNICEF-WHO-World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP). The authors alone are responsible for the views expressed in this article and they do not necessarily represent the views, decisions or policies of the institutions with which they are affiliated.
Competing interests None declared.
Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement The article describes development and application of a framework. The tables generated regarding study design may constitute data and are included in the published manuscript.