Article Text
PDF
Abstract
Objective To systematically identify and describe approaches to prioritise primary research topics in any health-related area.
Methods We searched Medline and CINAHL databases and Google Scholar. Teams of two reviewers screened studies and extracted data in duplicate and independently. We synthesised the information across the included approaches by developing common categorisation of relevant concepts.
Results Of 44 392 citations, 30 articles reporting on 25 approaches were included, addressing the following fields: health in general (n=9), clinical (n=10), health policy and systems (n=10), public health (n=6) and health service research (n=5) (10 addressed more than 1 field). The approaches proposed the following aspects to be addressed in the prioritisation process: situation analysis/ environmental scan, methods for generation of initial list of topics, use of prioritisation criteria, stakeholder engagement, ranking process/technique, dissemination and implementation, revision and appeal mechanism, and monitoring and evaluation. Twenty-two approaches proposed involving stakeholders in the priority setting process. The most commonly proposed stakeholder category was ‘researchers/academia’ (n=17, 77%) followed by ‘healthcare providers’ (n=16, 73%). Fifteen of the approaches proposed a list of criteria for determining research priorities. We developed a common framework of 28 prioritisation criteria clustered into nine domains. The criterion most frequently mentioned by the identified approaches was ‘health burden’ (n=12, 80%), followed by ‘availability of resources’ (n=11, 73%).
Conclusion We identified and described 25 prioritisation approaches for primary research topics in any health-related area. Findings highlight the need for greater participation of potential users (eg, policy-makers and the general public) and incorporation of equity as part of the prioritisation process. Findings can guide the work of researchers, policy-makers and funders seeking to conduct or fund primary health research. More importantly, the findings should be used to enhance a more coordinated approach to prioritising health research to inform decision making at all levels.
- Public Health
- Health systems
- Health policy
- Health services research
- Review
Data availability statement
All data relevant to the study are included in the article or uploaded as online supplemental information.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Key questions
What is already known?
Although there is a growing number of reports on approaches to primary health research prioritisation, we are not aware of any systematic synthesis of that body of evidence irrespective of research topic, geographical or institutional setting.
What are the new findings?
We created a common framework of prioritisation criteria and stakeholder types that respectively captured all criteria and stakeholders mentioned in each of the 25 identified approaches.
Less than half of the identified approaches proposed involving potential users (eg, policy-makers, government and the general public) or incorporated equity-related criteria as part of the prioritisation process.
What do the new findings imply?
We provide specific suggestions for which approaches to consider when emphasis is on patients and public engagement, equity, a specific field of research, or the availability of time and resources.
Introduction
Health research can strengthen health systems, accelerate progress on the Sustainable Development Goals and improve population health.1–4 The past few years have witnessed increased global calls to make better use of health research in policy-making and practice.5–7
The global COVID-19 pandemic has reinforced the importance of appropriately identifying the health issues that must be prioritised for research.8–11 Hydroxychloroquine represents a notorious example of inappropriate research investment and duplication of efforts. While initially haled as a miracle drug for treating patients with COVID-19, the ensuing research showed that this drug is ineffective and potentially harmful.12 13 In spite of that fact, numerous primary studies continued to be conducted on the effectiveness and safety of hydroxychloroquine in treating COVID-19 patients.14–16
The large number of competing topics for health research is coupled with limited available resources. Therefore, prioritisation processes are increasingly recognised as essential for the optimal allocation of resources to areas of greatest need and impact, especially in resource-poor environments.17–20 In particular, priority setting can maximise the likelihood that potentially impactful research is funded,21 and that research outputs reflect the needs of a broad range of stakeholders.22 23 It could also ensure more efficient and equitable use of limited resources and less duplication of research efforts.24–26 This aligns with global efforts to reduce research waste and avoid duplication of research efforts.27
A systematic and transparent prioritisation approach to assist decision-makers and research funding agencies in making investment decisions is critical.28 Although there is a growing number of reports on approaches to primary health research prioritisation, we are not aware of any comprehensive synthesis of that body of evidence; previous systematic reviews on prioritisation for primary research did not specifically explore the approaches used, and were restricted to geographic areas such as selected high-income countries23 or selected low-income and middle-income countries.24 More recent reviews examined approaches and exercises conducted to prioritise topics or questions specifically for systematic reviews29 or practice guidelines.30 31
Therefore, the aim of this study was to systematically identify and describe approaches to prioritise primary research topics in any health-related area, irrespective of geographical or institutional setting. Findings will enable a better understanding of the landscape of approaches used to prioritise primary health research.
Methods
We conducted a scoping review on approaches to prioritise primary research topics in health related areas, addressing the following broad areas: steps of the development process of the prioritisation approaches; aspects proposed to be addressed in the prioritisation process; methods for generation of initial list of topics; prioritisation criteria and stakeholder involvement.
We opted for a scoping review, which is typically used to present ‘a broad overview of the evidence pertaining to a topic, irrespective of study quality, to examine areas that are emerging, to clarify key concepts and to identify gaps’.32 Scoping reviews are an ideal tool to convey the breadth and depth of a body of literature on a given topic and give clear indication of the volume of literature and studies available as well as an overview of its focus.33 In contrast to a systematic review, it ‘is less likely to seek to address very specific research questions nor, consequently, to assess the quality of the included studies’.34
We followed standard methodology and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) guidelines for reporting scoping reviews35 (online supplemental file 1). This study is based on a protocol available in online supplemental file 2.
Supplemental material
Supplemental material
Eligibility criteria
Type of study: We included all types of study designs except for commentaries, news, editorials, correspondences, letters to editors, viewpoints, abstracts and reviews. While we excluded reviews, we planned to assess for eligibility the studies that they included.
Scope: We included studies describing approaches for prioritising topics for primary research in any health-related area.
We considered ‘approach’ as an umbrella term for frameworks, checklists, models, and methods used to set health research priorities. The description of the approach should have been detailed enough to allow for reproducibility, practically using at least one section dedicated to that description. We excluded studies describing the output of a prioritisation exercise (ie, the priorities) without describing the approach. We excluded studies describing individual prioritisation items or criteria (eg, burden of disease and cost) but not describing a prioritisation approach or model. Additionally, we excluded studies that looked at ranking techniques (such as Delphi and nominal group techniques) in isolation of a broader prioritisation approach.
Health-related areas included clinical, public health, health service and/or health systems and policy research. We did not limit the review to any specific health topic. We excluded animal studies and studies on genetics.
We considered primary research as quantitative or qualitative research that requires the researcher to engage directly in primary data-gathering process as opposed to depending on already existing data (ie, secondary research). We excluded priority setting approaches focusing on healthcare or service delivery priorities (ie, not specific to research).
Setting: We did not limit study eligibility to any geographical setting (eg, low-income, middle-income or high-income countries) or prioritisation level (institutional, subnational, national, regional or international).
Search strategy
We searched Medline and CINAHL electronic databases up until January 2021. We developed the search strategy with the help of an information specialist. The search combined various terms for health prioritisation and included both medical subject headings and free-text words. We did not restrict the search to any dates or languages. The detailed search strategy is provided in online supplemental file 3. We also manually searched Google Scholar as well as screened the reference lists of included studies and other relevant reviews to retrieve additional studies.
Supplemental material
Study selection
We completed the selection process in two stages:
Title and abstract screening: Three teams of two reviewers used the above eligibility criteria to screen titles and abstracts of identified citations in duplicate and independently for potential eligibility. They obtained the full texts for citations judged as potentially eligible by at least one of the two reviewers.
Full-text screening: The same three teams of two reviewers screened the full texts in duplicate and independently for eligibility. They resolved disagreement by discussion or with the help of a third reviewer when consensus could not be reached. They used a standardised and pilot-tested screening form.
Prior to proceeding with the selection process, we conducted two rounds of calibration exercises using a randomly selected sample of 100 citations for the first round, and a random sample of 50 citations for the second round. The calibration exercise allowed us to pilot the eligibility criteria to ensure they are applied in the same way across reviewers, thus enhancing the validity of the selection process.
Data abstraction
Two teams of two reviewers abstracted data from eligible studies in duplicate and independently using a standardised and pilot-tested data abstraction form. Disagreements were resolved by discussion, and when needed, with the help of a third reviewer. We conducted a calibration exercise on a random sample of three studies to ensure the data abstraction variables are clear and interpreted in the same way across reviewers, thus enhancing the validity of the data abstraction process.
We abstracted the following information from each included approach:
General characteristics: authors; location; year of publication; name of the approach; lead entity; target audience; field (eg, clinical, public health or health systems); level of prioritisation (institutional, subnational, national, international); funding agency; output of prioritisation and type of publication.
Steps of the development process of the prioritisation approaches, we used the abstracted data to create a common categorisation of the steps used in the development process (eg, use of a pre-existing framework/approach; literature review; consensus building; stakeholder input; pilot-testing). We collected this information as it reflects the thoroughness of the development process.
Aspects proposed to be addressed in the prioritisation process; we used the abstracted data to create a common categorisation of the aspects proposed by the approaches (eg, situation analysis/environmental scan; methods for generating initial list of research topics; use of prioritisation criteria; stakeholder engagement (types of stakeholders proposed to be involved and the role of the different proposed stakeholders); ranking process/technique; dissemination and implementation; revision or appeal mechanism; and monitoring and evaluation). We collected this information as it reflects the breadth of the aspects of prioritisation covered by the approach.
We did not conduct a formal assessment of the risk of bias within or across studies given the descriptive nature of the included studies Furthermore, we are not aware of, and could not identify tools to critically appraise the types of studies retrieved.
Data synthesis
Given the nature of data, we synthesised the findings in a semiquantitative way. We used the abstracted data to come up with common categorisations of relevant concepts (eg, steps of the development process of the approach, aspects proposed to be addressed in the prioritisation process, prioritisation criteria, type and level of stakeholder involvement, methods for generation of initial list of topics), using an iterative process of review and refinement. As part of this process, the content of each study was analysed at least twice; once when drafting the initial categories, and after producing an advanced draft. Throughout this process, team members with subject expertise were consulted to validate categorisation decisions and discuss emerging concepts. We reported the results in both narrative and tabular formats.
The concepts we addressed in our analysis were the following (with the analytical approach that we followed included in brackets):
Steps of the development process of the prioritisation approaches (content analysis).
Aspects proposed to be addressed in the prioritisation process(content analysis).
Methods for generation of initial list of topics (descriptive analysis).
Prioritisation criteria: we used as an initial list of criteria derived from a framework of prioritisation criteria for evidence synthesis topics.29 Two reviewers (RF and ND) independently matched the criteria reported in the included studies to the initial list of criteria derived from the framework. The criteria for which consensus could not be reached were independently reviewed and validated by a third reviewer (AE-H). We subsequently revised the list of criteria to capture those not already captured and drop those that may not apply. Multiple meetings were held to finalise the list of prioritisation domains and criteria.
Stakeholder involvement: we adopted the categories we developed for a recent systematic review on prioritisation for evidence synthesis29 which is based on the 7Ps framework36 as a starting point. We subsequently revised the list of categories to capture those not already captured; for stakeholder roles, we applied content analysis; one reviewer (AE-H) generated an initial list of stakeholder roles from the included studies (based on the data on stakeholder roles abstracted independently by two reviewers). Another researcher (RF) verified the resulting list to improve its clarity and relevance. The two reviewers then met to finalise the list of stakeholder roles through discussion and consensus.
We concluded the results section with a subsection on considerations relevant to selecting an approach.
Patient and public involvement
Patients and the public were not involved in the design and conduct of this scoping review. However, findings have implications for patient and public involvement in the prioritisation of topics for primary health research.
Results
Study selection
Figure 1 shows the study flow diagram which summarises the selection process. Out of the 44 392 citations identified, we included 30 articles reporting on 25 approaches, with each of 2 approaches reported in 2 articles37–40 and 1 approach reported in 3 articles.41–43 The articles were published between 1997 and 2020 inclusive. Seven of the included articles were obtained from the reference list/bibliography.37 44–49 We excluded 1623 articles at the full-text screening phase for the following reasons: not study design of interest (n=135); focus is not on a prioritisation approach or tool (n=1043); focus is not on research (n=432); and focus is not on public health, health service, health systems or clinical fields (n=13) (see online supplemental file 4 for reasons for exclusion).
Supplemental material
Study selection process.
General characteristics of the prioritisation approaches
Table 1 shows the general characteristics of the 25 identified approaches (see online supplemental file 5 for detailed overview and description of each approach). The majority of the approaches originated from high-income countries, with government as the main funding source (n=10, 40%). Eighteen of the 25 (or 72%) approaches were published in peer reviewed articles. The included approaches addressed the following fields: health in general (n=9, 36%); clinical (n=10, 40%), health policy and systems (n=10, 40%); public health (n=6, 24%) and health service research (n=5, 20%); Ten of the approaches addressed more than one field. The approaches are applicable to the following levels of prioritisation: national (n=20, 80%) international (n=7, 28%), institutional (n=7, 28%), regional (n=5, 20%) and subnational (n=7, 28%); ten of the approaches addressed more than one level of prioritisation. As for prioritisation outputs, these included research topics (n=10, 40%), specific research questions (n=6, 24%), or research themes (n=1, 4%); in nine of the approaches, the output was not specific. Seventeen of the approaches proposed ranking the priorities and the remaining eight proposed listing them (without ranking). Only one approach (reported by three papers) also proposed prioritising outcomes for the priority research questions.41–43 Seven of the approaches had an explicit focus on promoting patient participation,45 46 50–54 and three (reported by four papers) incorporated an equity dimension to health research priority setting.38 49 55 56
Supplemental material
General characteristics of the 25 included approaches
Development process of the prioritisation approaches
Table 2 provides a description of the steps followed for the development process of the different prioritisation approaches. In three approaches (reported by four papers), the development process was not described.39 40 57 58 We categorised the steps as follows: use of a pre-existing approach, literature review, consensus building, stakeholder input and pilot-testing. The most frequently reported steps used in the development process were reviewing the literature (n=14, 64%) and building on a pre-existing approach (n=14, 64%). Eleven (or 50%) of the approaches were pilot-tested.
Steps of the development of the prioritisation approaches (N=22)
Eight of the approaches (reported by nine studies)38 50 59–64 covered more than half of the steps identified in the development process, with one following all of the steps in the development process.59
Aspects to be addressed in the prioritisation process
Table 3 shows the aspects to be addressed when prioritising primary research topics, as proposed by the different approaches. The vast majority of approaches (n=23, 92%) included the involvement of stakeholders as one aspect of prioritisation. Most approaches also proposed the following as key aspects to be addressed in the prioritisation process: methods for generation of initial list of topics (n=19, 76%), use of prioritisation criteria (n=18, 72%), description of ranking process/technique (n=18; 72%) and dissemination & implementation of prioritisation outputs (n=16, 64%). Less than half of the approaches proposed monitoring and evaluation (n=10; 40%).
Aspects to be addressed in the prioritisation process, as proposed by the approaches (N=25)
Five approaches (reported by six papers) covered all aspects of prioritisation, and were, thus, considered the most comprehensive and detailed.39 40 45 52 62 63
Methods proposed for generating initial list of topics
Nineteen approaches proposed methods for generating an initial list of topics for subsequent prioritisation (see table 4). The most frequently used method was seeking input from stakeholders (n=16, 84%) followed by reviewing the literature (n=9, 47%) and assessing research gaps from existing systematic reviews (n=7, 37%). The majority of approaches (n=15, 79%) used more than one method to generate initial list of topics. Four approaches relied on stakeholder inputs alone to generate initial list of topics.37 52 54 55
Methods proposed for generating initial list of topics (N=19)
Prioritisation criteria
Fifteen of the 25 identified approaches (60%) proposed a list of criteria for determining research priorities. There was a total of 135 mentions of criteria across the approaches (range: 3–28). In some of the approaches, there was flexibility in terms of using the prioritisation criteria, that is, criteria were presented as a menu option to select from depending on context. In four approaches (reported by five papers), the proposed criteria constituted different steps of a process or model for prioritisation.23 38 44 58 Three approaches gave weight to the criteria,37 59 63 while two approaches (reported by three papers) proposed weighting of criteria as optional.39 40 62 The remaining approaches did not suggest weighting the criteria. Three approaches proposed that the criteria need to be applied by research experts.37 46 58
We attempted to match the 135 criteria to a published framework of 25 prioritisation criteria classified into 10 domains (online supplemental file 6). In the process, we merged the two domains ‘existing systematic reviews’ and ‘existing primary studies’ into a single domain ‘existing research base’, and revised the criteria accordingly, in alignment with the included studies for this review. Table 5 shows the classification of the identified 28 prioritisation criteria according to the new framework, clustered in nine domains: (1) problem-related considerations; (2) practice considerations; (3) existing research base; (4) amenability to research; (5) urgency; (6) interest of the topic at different levels; (7) implementation considerations; (8) expected impact of applying evidence and (9) ethical, human rights and moral considerations. The criterion most frequently mentioned by the identified approaches was ‘health burden’ (n=12, 80%), followed by ‘availability of resources’ (n=11, 73%), and ‘economic outcomes’ (n=10, 67%). Only one approach (reported by two papers) incorporated more than half of the 28 criteria listed in the framework.39 40
Supplemental material
Framework for prioritisation domains and criteria (N=15)
The remaining approaches that did not propose using criteria relied on a variety of processes to rank priorities (eg, Delphi technique, nominal group techniques, ranking using a 10-point scale, simple voting), the most common being the Delphi technique.41–43 50 One approach presented a range of ranking techniques, including comparison in pairs; anchored rating scale; Hanlon method and Essential National Health Research (ENHR) method.47 Another approach highlighted several different methods to decide on priorities that broadly fall into two groups: consensus based approaches and metrics based approaches.62
Stakeholder involvement
All but three approaches proposed involving stakeholders in the priority setting process44 47 58 (table 6). The stakeholder category most frequently proposed for involvement was ‘researchers/academia’ (n=17; 77%), followed by ‘healthcare providers’ (n=16, 73%). While slightly more than half of the approaches proposed involving ‘patients and their representatives’ (n=13, 59%), less than half proposed involving ‘members of the public’ (n=9, 41%), ‘caregivers’ (n=7, 32%), ‘NGOs and adovacy groups’ (n=8; 36%) or ‘government/policy-makers’ (n=10; 45%) in the prioritisation process.
Types of stakeholders proposed to be involved in prioritising primary research topics (N=22)
We identified nine distinct roles for stakeholder involvement in the priority setting process (table 7): executive committee/coordination; theme identification phase; establishment of initial list of topics/questions; refinement of topics/questions; prioritisation/ranking of topics/questions; selection of prioritisation criteria and weighting method; validation of prioritisation outputs; dissemination and process evaluation. Across the different stakeholder categories, the most commonly mentioned role was prioritisation/ranking of topics/questions followed by establishment of initial list of topics/questions and theme identification.
The roles for the different types of stakeholders proposed to be involved in prioritising primary research topics (N=25)
Eleven approaches described stakeholder recruitment methods, and these ranged from announcement in journal and newspapers, on website, by letter and distribution of brochures; to use of emails and established contacts; mapping stakeholders; checklist for identification of stakeholders; and organisational and personal contacts.37 45 47 50 51 63 Additional methods to recruit representatives of patients and the general public included social media (Twitter, Facebook), radio ads and leveraging existing community-based partnerships. Stakeholders were engaged both via online platforms (eg, online surveys, email discussions, teleconference) and in-person (eg, workshops, smaller meetings) in eleven approaches (reported by 14 papers).37 39–43 45 51–53 61 63–65
Considerations relevant to selecting an approach
Below, we provide considerations relevant to selecting a prioritisation approach when the emphasis is on one of the following: patients and public engagement, equity, a specific field of research, or the availability of time and resources.
Groups seeking to engage patients and the public in prioritisation would benefit from adopting one of the following seven approaches that have highly structured patient and public engagement planning activities45 46 50–54 : the James Lind Alliance (JLA) method (UK), Listening Model (England and Canada), Dialogue Model (Netherlands), The PRioritiEs For Research project informed by the Dialogue model (Canada), Rare Disease Research Partnership approach (Ireland), storytelling approach to identify patient-centred research priorities (USA) and the Approach informed by Patient-Centered Outcomes Research Institute framework (USA).
Priority setting exercises that value the principles of equity should consider one of three approaches (reported by four papers) that explicitly incorporate an equity dimension to health research priority setting.48 49 55 56 The deep inclusion model developed by Pratt et al has been developed for use where research priority-setting is conducted in the context of power inequalities.55 In the three-dimensional (3D) Combined Approach Matrix, the equity dimension facilitates comparison of different social groups in relation to particular health-related or health systems-related problems, ultimately resulting in informed policy decisions that are aimed at improving not only the average level of health, but also its distribution and hence, equity. Social groups can be defined on the basis of gender, income level, race or ethnicity, religion or sexual orientation, depending on the context.48 The Research Priorities of Affected Communities protocol is designed to elicit research questions and achieve consensus on priority research topics directly from members of members of under-represented communities that bear the burden of health disparities related to the health condition of interest. The method is based on a pedagogical framework of Research Justice that seeks to equalise the political power and legitimacy of knowledge generation.49 56
Additionally, the particular field of research should be considered when selecting the priority setting approach to use. While many of the existing approaches (9 approaches reported by 11 papers) address health in general,38–40 47 48 50 57 61–63 66 others are applicable to more specific fields: clinical (10 approaches reported by 12 papers),41 43–45 49 51 53 54 56 59 60 65 67 health systems and policy (10 approaches reported by 11 papers),37 44 49 51–56 60 65 public health (6 approaches reported by 7 papers)37 49 53 54 56 58 65 and health service research (5 approaches reported by 6 papers).46 49 56 58 59 64 For instance, the disability-adjusted life-year-based amended model adopts a clinical lens to define research priorities, which excludes a health systems perspective.44 Similarly, the JLA method has a narrow focus on clinical settings and is overly biased to treatment needs, as opposed to system needs.45 On the other hand, the Seven ‘I’s model does not explicitly encompass a specific disease or injury component as a criterion for priority setting as doing so would focus attention on the ‘wrong end of the health outcomes and risk factor spectrum’; the model encourages the link to population health gains.58
In terms of time and resource constraints, multicomponent approaches have been found to be resource intensive, requiring the involvement and coordination of many participants across multiple stages. This has been described for the 3D Combined Approach Matrix,48 the Child Health and Nutrition Research Initiative37 and the eight-step process for identifying and prioritising clinically important research needs.41 43 67 In contrast, the developers of the CAHTA method describe it as a relatively agile, low-cost, participatory process that allows for priority-setting over a wide range of topics.59
Discussion
We systematically reviewed the literature on prioritisation approaches for primary research topics in any health-related area. We identified and described 25 prioritisation approaches. The majority of approaches addressed clinical and health policy and systems research topics, were applicable at the national-level, were published by independent researchers and targeted a broad range of stakeholders.38 45 46 50 51 55 None of the included studies reflected on the additional applicability of the identified approaches to other types of health research (eg, evidence synthesis).
There were variabilities in the steps adopted to develop the approaches and the aspects proposed to be addressed in the prioritisation process. Eight approaches (reported by nine papers)38 50 59–64 covered more than half of the steps identified in the development process. Five approaches (reported by six papers)39 40 45 52 62 63 covered all aspects of the prioritisation process (and are, thus, considered the most comprehensive and detailed), namely: situation analysis/environmental scan, methods for generation of initial list of topics, use of prioritisation criteria, stakeholder engagement, ranking process/technique, dissemination and implementation, revision or appeal mechanism, and monitoring and evaluation. Stakeholder involvement and the use of prioritisation criteria represented key aspects of most of the prioritisation approaches.
There was also wide variation across approaches in the prioritisation criteria. To address this, we synthesised the information across the included approaches by developing common categorisation of relevant concepts. This resulted in a common framework of 28 prioritisation criteria clustered in 9 domains. Equity was an infrequent dimension of the priority setting, as reflected by the low number of studies incorporating equity-related prioritisation criteria.
We also developed a common categorisation of 13 stakeholder types and nine distinct stakeholder roles in the priority setting process. The most commonly proposed stakeholder type was researchers/academia followed by healthcare providers, and the most commonly mentioned stakeholder role was prioritisation/ranking of topics/questions.
Despite increased calls for involving research users in the priority setting process, less than half of the approaches involved governments and policy-makers. Engaging governments and policy-makers in research priority-setting exercises can enhance alignment of research production to policy priorities and needs, which in turn, can increase the relevance and likelihood of utilisation of research to inform decisions.19 68 While slightly more than half of the approaches proposed involving patients and their representatives, less than half proposed involving members of the public, caregivers or NGOs and advoacy groups in the prioritisation process. The rising trend in patient involvement may reflect the growing number of patient-centred approaches, particularly within the most recent years. Patient and public participation in priority setting makes research tangible, relevant and valuable for patients and their relatives, enhances the legitimacy and fairness of decision making,69 improves trust and confidence in the health system70 and strengthens the quality of decision making.71 72 Without such engagement from the earliest stages, researchers and healthcare providers may ultimately miss the priority needs of the end users. While the involvement of a wide range of stakeholders can increase the legitimacy, transparency and acceptability of the identified priorities, it also raises challenges in terms of capacity, coordination, communication and resources.63 73 Unfortunately, none of the identified approaches examined these issues in-depth.
Although the global COVID-19 pandemic has reinforced the importance of health research prioritisation, none of the identified approaches focused on identifying research topics or questions in the context of emergencies or public health crisis. Existing priority setting related to COVID-19 pandemic focused largely on prioritisation of patients or healthcare interventions (eg, medications) as opposed to research topics or questions. A recent study on informing Canada’s health system response to COVID-19 conducted a rapid-cycle priority identification process to identify seven COVID-19 priorities for health services and policy research. However, no sufficient details were reported on the methodology applied to generate the priorities.74
To our knowledge, this is the first study to comprehensively review approaches for prioritising primary research topics in any health-related area, irrespective of geographical or institutional setting. Previous reviews have mainly focused on specific geographic settings23 24 or specific research types.29 In line with our findings, these reviews reported variabilities in priority setting methodologies with inconsistent application of methods and outcomes generated. On the other hand, our findings provide a comprehensive description of approaches for primary research topics in any health-related area, with a more in-depth analysis of relevant characteristics such as the steps, criteria and stakeholders for prioritisation, which enables a better understanding of the landscape of approaches used to prioritise primary health research.
Strengths and limitations
Strengths of our methodology include applying a rigorous and transparent process, following standard methods for reporting scoping reviews and including different types of study designs and settings. We also followed an iterative process of review and refinement to create a common framework of prioritisation criteria and stakeholders that respectively captured all criteria and stakeholders mentioned in each of the 25 identified approaches. This denotes progress toward standardising the terminology for prioritisation and enhancing the clarity of criteria for decision making.
Despite our attempt to identify all priority setting approaches for primary health research topics, we could have missed on potentially relevant information, particularly those in specialised data sources (eg, organisational websites). Our review did not seek published data on the implementation of the identified approaches. Additionally, no assessment of risk of bias was conducted given no tool is available for this type of studies; however, this is consistent with the scoping review methodology.35
Implications for practice and policy
The findings of this scoping review can guide the work of researchers, policy-makers, funders and other stakeholders in the field of health research. Those involved should select the approach that best fits their needs, taking into consideration the purpose of priority setting as well as available resources and time constraints. For example, groups can consider our list of prioritisation criteria as a menu of options to select from, as deemed appropriate to the context, with considerations to incorporate equity-related criteria. Efforts should also be invested to ensure greater participation of potential users (eg, policy-makers, government and the general public) as part of the prioritisation process.
Application of the approaches can help support evidence-informed policy-making by aligning research production to policy priorities and needs. They can also help avoid research waste by directing limited resources to areas of highest priority and impact (eg, several of the identified approaches proposed the assessment of research gaps from systematic reviews and subsequent use of such information to generate initial list of primary research topics for prioritisation in order to avoid duplication of work). This is especially relevant in the context of low-income and middle-income countries where resources are already scarce and capacity for research production is limited and often misaligned with policy priorities and needs.
Implications for research
Further rigorous research is needed to examine the feasibility, effectiveness and transparency of several of the identified approaches. This kind of research would allow a better understanding of the potential barriers and facilitators to prioritisation using the existing approaches. It is also essential to evaluate the impact of those approaches on research agenda setting and broader health outcomes. Future work should use the findings of this review as well those of similar reviews addressing other types of health research (eg, evidence synthesis and guideline development) as the building blocks to produce an overarching approach to prioritising topics across the health research spectrum. The ultimate goal should be to inform the decision making of different stakeholders, including government, organisations, professionals and citizens.
Additionally, given the majority of the approaches were developed by researchers from high-income countries, future research can assess the applicability of the approaches in middle-income and low-income countries. Similar research is needed on the applicability and adaptability of the identified approaches beyond the health sector. For example, there is growing interest in approaches to prioritise research topics during pandemics where resources, infrastructure and government capacity to respond are particularly constrained.8–11 The applicability of the identified approaches in the context of emergencies and crises warrants further exploration.
More rigorous research is also needed to assess effective ways of involving stakeholders in different aspects of prioritisation with a focus on promoting greater participation of potential users (eg, policy-makers, government and the general public) including better reporting of operational details of stakeholder engagement. Findings also highlight the need for more research on how to promote equity in priority-setting.
More generally, and given the variability in the identified prioritisation approaches, there is a need for guidance for developing priority-setting approaches and reporting their findings.
Data availability statement
All data relevant to the study are included in the article or uploaded as online supplemental information.
Ethics statements
Patient consent for publication
Ethics approval
Not applicable.
Acknowledgments
The authors would like to acknowledge Ms. Aida Farha for her help in developing and validating the search strategy.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Handling editor Stephanie M Topp
Contributors EAA and RF conceived and designed the study. FE-J contributed to the conceptualisation of the study. RF coordinated the study throughout. RF, EAA, AE-H and TL ran the search. RF, ND, AE-H, RH, HB, LBK, LCL, GA, IK, SB, AY and TL conducted the study selection processes (title and abstract screening followed by full text screening). RF, ND, AE-H. RH, HB extracted the data. RF, ND, AE-H and EAA. analysed and interpreted the data. RF wrote the first draft of the manuscript with EAA. ND contributed to writing of the manuscript. All authors critically revised the manuscript and approved the final version. EAA will act as guarantor.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.