Article Text
PDF
Abstract
Dialogue and exchange between researchers and policy personnel may increase the use of research evidence in policy. We piloted and evaluated a programme of formalised dialogue between researchers and provincial health policymakers in South Africa, called the buddying programme. An external evaluation examined implementation and short-term impact, drawing on documents, in-depth interviews with policymakers, a researcher buddies focus group and our own reflection on what we learnt. We set up buddying with seven policymakers and five researchers on six policy questions. Researchers knew little about policymaking or needs of policymakers. Policymakers respected the contact with researchers, respected researchers’ objectivity and appreciated the formalised approach. Having policymaker champions facilitated the dialogue. Scenarios for policy questions and use were different. One topic was at problem identification stage (contraceptives and HIV risk), four at policy formulation stage (healthy lifestyles, chronic illness medication adherence, integrated care of chronic illness and maternal transmission of HIV to infants) and one at implementation stage (task shifting). Research evidence were used to identify or solve a policy problem (two scenarios), to legitimise a predetermined policy position (three scenarios) or the evidence indirectly influenced the policy (one scenario). The formalised dialogue required in this structured buddying programme took time and commitment from both sides. The programme illustrated the importance of researchers listening, and policymakers understanding what research can offer. Both parties recognised that the structured buddying made the dialogue happen. Often the evidence was helpful in supporting provincial policy decisions that were in the roll-out phase from the national government.
- evidence-informed policy
- policymaking
- research use
- engagement
- dialogue
- decision making
- knowledge translation
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
- evidence-informed policy
- policymaking
- research use
- engagement
- dialogue
- decision making
- knowledge translation
Summary box
Exchange between policymakers and researchers may increase evidence uptake in policymaking. We piloted a structured linkage programme in a province in South Africa and successfully increased one to one dialogue between researchers and policymakers.
We learnt that researchers need to learn more about policymaker world; and subnational policymakers often found evidence useful in understanding and rolling out decisions made at national level.
Building relationships and dialogue took time and commitment from both sides.
Introduction
Evidence-informed health policy (EIHP) is characterised by systematic and transparent approaches to access, appraise and use evidence as inputs to decision-making processes.1 Its application is influenced by the complex nature of policymaking, as well as policymakers’ access to and capacity to use evidence. Research evidence is only one of many potential inputs into complex policymaking processes, with other forms of information, interests, context and institutional factors vying for policymakers’ attention.2 3 However, even when policymakers want to consider evidence, they report barriers to finding and using it, including limited time4 and skills to find and appraise research evidence,5 unavailability of research when it is required,5 6 irrelevance of research5 and presentation in formats that decision makers cannot readily use.
Alternatively, facilitators and interventions to support and increase EIHP have been identified, targeting policymakers, researchers, exchanges between them and their environment.5 7 These strategies are sometimes referred to as “producer-push” strategies (eg, producing summaries of systematic reviews,8 “user-pull” strategies (where policymakers seek evidence) and “linkage and exchange”.9 Some analyses have flagged that researchers are at times “theoretically naïve”, assuming that policymakers do not use research evidence, and focus on a dynamic of “getting research evidence into policy”, whereas what is needed is academics that understand the policy process better.3 One approach to improve understanding on both sides relates to interpersonal relationships and communication between research users and producers,5 thus fostering knowledge broker strategies10 and application of network science to identify opportunities for strategic linkages.11 12
In our role as researchers in South Africa, working in evidence synthesis and Cochrane, we planned a 2-year project to evaluate a structured linkage approach where researchers were “buddied” or partnered with policymakers (one-to-one) to increase dialogue, with the intention of increasing demand for and uptake of systematic review evidence. Systematic reviews are well recognised as internally valid evidence sources13 and efficiencies of their use in policymaking have been argued extensively.9 13 Policy BUilding Demand for evidence in Decision making through Interaction and Enhancing Skills (BUDDIES), funded by the WHO Alliance for Health Policy and System Research,14 was implemented in full in South Africa and partly in Cameroon. This paper describes implementation in the Western Cape Province, South Africa.
As a baseline, we interviewed health policymakers in the Western Cape to understand policymaking processes, how research evidence may contribute, enablers and barriers to demanding and using evidence during policymaking (reported by Naude 2015).15 Similar to a national-level study,16 participants outlined complex processes, with research not playing a key role (other drivers include personal expertise, costs and feasibility and preferences of managers). In theory, research evidence can be used at various policy stages: in defining the problem, assessing policy and programme options and identifying implementation considerations.9 Policymakers however face various barriers to using research.15 These barriers to using research, like those found by Oliver and colleagues,5 call for pragmatic solutions, and some have used rapid response services,17 including rapid reviews18 or communities of practice.14 Our objective was to pilot and evaluate a novel intervention to build relationships (termed buddying) between researchers and policymakers to increase the use of evidence in provincial health policy decisions.
What did we do?
Based on our baseline research,15 we designed the buddying programme, which was implemented in 2014 for 6 months (figure 1). We implemented the programme in the Western Cape and conducted an external mixed-methods evaluation.
Summary of buddying approach.
South Africa is a constitutional democracy and consists of three levels of government—national, provincial and local. There are nine provinces, each with its own provincial legislature. Provincial governments, such as the Western Cape, are bound by laws and policies passed at national level but can also develop their own laws and policies within this framework to suit their specific needs. Health infrastructure and services in the Western Cape are generally regarded as better than in most other provinces.
The buddying programme linked provincial policymakers one-to-one with local researchers. This was structured as a programme with a principle of working together on an equal basis, instead of mentorship where an experienced, highly regarded person (the mentor) usually guides another individual (the mentee) in his/her development. Policymakers working in public health, nutrition, sexually transmitted infection (STI) and HIV-identified relevant policy questions during baseline,15 and we conducted a workshop for this group on finding, assessing, interpreting and using systematic review evidence. These questions were prioritised through discussion between policymakers and researchers, and became the basis for buddying programme.
Researcher buddies were selected for their experience and expertise in evidence-based healthcare and policy. They were matched with policymakers aligned with their content knowledge for example, for the nutrition-related question, the buddy with nutrition background was linked to the relevant policymaker. While some researchers and some policymakers knew each other personally, there had been no real dialogue between them about policy and research evidence prior to the study. Researcher buddies participated in meetings, conversations, discussions, which were not taking place at baseline. Each researcher buddy initiated and facilitated the dialogue with the policymaker, but also discussed the best approach in relation to the topics with the researcher buddies group (all researchers involved in the programme). The researcher buddies participated in policy workgroups, presented at policy meetings, called and/or emailed their policymaker buddy and related teams from the Western Cape Department of Health. The researcher buddies used a dedicated online website and monthly meetings to ensure consistency of approach, to exchange experiences, to share resources and to reflect on progress and document interactions, reflections and engagements. Some researcher buddies were paid for their role as staff members on the programme. Researcher buddies did not receive formal communication training but were encouraged to use each other as resources and support.
An individual with no previous involvement in the programme (JCS) carried out an external mixed-methods evaluation towards the end of the programme. The evaluation examined how the approach was implemented, barriers and successes during its implementation and uptake; learning strategies developed during this process and the short-term impact of the programme on policymakers’ use of research evidence to inform their decision making. Data were collected in several ways—document review of policy and programme documents and researcher buddies’ structured reflections; in-depth, semistructured interviews with policymakers and a focus group discussion with researcher buddies (online supplementary additional file 1: interview guides). All researcher buddies participated. JS conducted the interviews, audio recorded them and an observer took notes. Following interviews, notes were expanded with the aid of the audio recordings. Expanded notes were coded in AtlasTi using a predefined codebook based on the evaluation questions and knowledge translation (KT) theory.2 3 19 Coded data were analysed, with an emphasis on emergent themes, negative data and triangulated across the multiple data sources. Neither the interviewer nor note taker were members of the implementation team. The interviewer identified herself as an “independent evaluator” at the start of each interview.
Supplemental material
What did we find?
Seven in-depth, semistructured, in-person interviews and one telephone interview were done with policymakers participating in the programme in Cape Town. The focus group discussion included five researcher buddies.
Document analysis showed that six policymaker-driven questions were tackled by the researchers buddies. Scenarios for evidence requests were different: for some questions, evidence was sought to endorse existing policies and for others, to inform new policy development (table 1). One topic was at problem identification stage (depo contraceptives and HIV risk), four at policy formulation stage (healthy lifestyles, chronic illness medication adherence, integrated care of chronic illness and maternal transmission of HIV to infants) and one at implementation stage (task shifting). We observed various uses of evidence across the cases according to types of use defined by Beyer and Trice.20 Three scenarios of symbolic use (use legitimised a predetermined policy position), two of instrumental use (specific, direct use of research evidence to identify or solve a policy problem) and one of conceptual use (the evidence indirectly influenced the policy) (table 1).
For example, HIV/AIDS and its treatment is a perpetual issue on the policy agenda in Western Cape Province. To extend antiretroviral coverage, shifting responsibility for diagnosis and care from doctors to nurses was an emerging policy issue, leading one policymaker to request evidence around task shifting (table 1). After defining the question in a one-to-one meeting, the researcher buddy prepared a summary and sent to the policymaker with the full Cochrane review on the topic in adults and found another review of task shifting in children. On request of the policymaker, three researcher buddies joined a policy forum, provided a formal presentation, handouts and participated in discussion of the evidence. At that stage, the policy for task shifting had already been prepared, but not rolled out, and the policymaker and their colleagues reported relief that the evidence supported the current draft policy.
Another policy considered was about improving healthy eating and exercise (table 1). The researcher buddy identified existing reviews and described this evidence at meetings “to start sensitising (the policymaker) and others about the role of systematic review evidence in decision making.” As interventions had already been decided on, it limited the possibility of evidence being used instrumentally20 (ie, direct use of research evidence to identify or solve a policy problem). However, it helped with design of some specific components, such as the catering guidelines. The researcher continued engaging as a core member of a task team on wellness.
For both, the policy questions on hormonal contraception and HIV acquisition and adherence to chronic medication (table 1), researcher buddies and policymakers worked together to prepare policy statements. Changes in their government position or time away from work were realities with a system of one to one partnerships. These delayed or simply led to discontinuation of the collaborative work.
Interviewed policy buddies reported that they recognised and valued evidence-informed policymaking. Lack of time, limited or no access to research databases and limited capacity to interpret research evidence, all highlighted during the situational analysis,15 made the buddying programme an opportunity for policymakers to access relevant research evidence from the researcher buddies.
Policymakers found evidence particularly useful when it backed up decisions that were not universally agreed on by all policy stakeholders, or to provide confidence when challenging existing policies. The researchers most often presented research evidence describing the effectiveness of different interventions and strategies. Policymakers, on the other hand, were curious to see other policy models, and wanted evidence on operational, implementation and organisational strategies for interventions.
The policy makers reported the researcher buddies as thoughtful and independent. Individual researcher buddies varied in terms of how much time they spent working with and responding to questions by policymakers. Some researcher buddies reported sending evidence, or question clarifications, and never hearing back, while others established further linkages at both provincial and national levels.
Case studies from the buddying programme (each representing one researcher–policymaker engagement)
What did we learn?
While the mantra of EIHP may be growing in popularity,5 the problem of “privileging academics’ research priorities”3 remains a problem. We piloted the buddying programme to build relationships between policymakers and researchers5 and to help researchers and policymakers work together.11 12 At the outset to the programme, researcher buddies were somewhat naïve. They believed that health decision makers were not using systematic reviews of research evidence sufficiently, and that the programme would help policymakers to find and use them, in linear, “getting evidence into policy” approach. However, the programme helped to reflect on these assumptions (box 1 and box 2).
Lessons learnt
Researchers have a lot to learn about policymaking.
Policy questions are not only about effects—other types of questions are equally important.
Policymakers respect researchers’ objectivity.
A structured relationship opened the door.
Responding to policymakers is time consuming and requires flexibility.
Having champions appears to facilitate the dialogue.
Having a support network helped researchers respond.
Continued researcher–policymaker engagement to promote evidence-informed policymaking cannot be sustained without dedicated time and resources
Illustrative quotes of lessons learnt
‘As a buddy, I learnt that we have much to learn about decision-making in our government environment.’ (Researcher buddy 5)
‘It takes a lot of time just to talk and build relationships. One week, for example, I felt I didn’t get any work done, because all I did was talk. But actually I did because I was building relationships. Just talking does achieve a lot, but it takes time.’ (Researcher buddies’ Focus group discussion)
Buddying taught me that when engaging with policymakers about evidence, one needs to be flexible and depending on the nature and format of the engagement, one needs to try and find the most feasible mode of communicating the evidence. This often means that you need to not be constrained by a stringent “one size fits all” theoretical approach, but rather ‘think on your feet’ and be less prescriptive about how evidence should inform decisions. (Researcher buddy 5)
The policy-maker and researchers speak different languages, particularly if not both familiar with evidence-informed policy-making concepts. (Researcher buddy 3)
We don’t have their capacity/skills and they don’t have ours. The linkage is the key to make the process smoother. (Researcher buddies Focus group discussion)
Researchers have much to learn
As researchers, we started the programme believing all health workers should be using systematic reviews, and that the programme would simply help policymakers to find and use them. Through the programme, we realised that systematic review evidence is one input into policymaking, and other factors such as cost, feasibility and politics play important roles. As researcher buddies, we need to learn more about the policymaking world. Linking with policymakers helped us to understand the policymaking processes and how to be more helpful, and flexible, when communicating research.
Policy questions are often not about effects
While the researcher buddies focus was on effects of interventions and often used the Population, Intervention, Comparison, Outcome “PICO” framework) to phrase clear questions, the questions from policymakers were not just about interventions and their effectiveness. Examples of such questions—Why are males not coming for medical male circumcision to reduce HIV and STI infections? Does service redesign lead to an integrated care pathway? This led us to realise that many of policymakers’ most pressing questions are complex, related to health systems or implementation, and thus may require new ways of reviewing and presenting evidence.
Policymakers respect researchers’ objectivity
Policymakers reported strong levels of trust for their researcher buddies, in large part due to their perceived objectivity and neutrality. ‘[Buddy] is somebody neutral who is an evidence specialist, [buddy] can verify what academics tell us.’ (Policymaker 7)
Buddying is time consuming and requires flexibility
Both parties reported challenges in scheduling time with each other. Policymakers would often reschedule planned meetings due to last-minute conflicts inherent to their roles. Researchers did not anticipate this and were not used to this. Researcher buddies were not prepared for the considerable amount of time it took to build trust and relationships. Researchers had to be flexible and adaptable.
Having policymaker champions facilitates the dialogue
The role of policy ‘champions’—those who would be invested in using evidence and incorporating the researcher into their work—in driving policies was clearly important and noted by both policymakers and researchers. Because the programme worked with individuals, the loss of a given individual, as happened in one case study, meant “you have to start again from scratch” (researcher buddy 6).
The researcher support network helped
Researcher buddies commented that they appreciated the opportunity to meet monthly with each other and troubleshoot. All researcher buddies reported exchanging evidence with each other during these meetings and helping each other address policymakers’ questions. In this way, the researcher buddies were fully networked.
The programme’s structured relationship opened the door
Policymakers liked the contact with the researchers, saying it opened a door and removed barriers to entry in asking researchers questions; and helped them structure time in their diaries to participate. The research team was from known units promoting evidence-based practices, which may have influenced how policy buddies responded. However, both researchers and policymakers noted that without the structured mechanism for engagement, they would be less likely to request assistance (policymakers) and would be less timely in their provision of assistance (researchers).
Conclusion
Barriers to finding and using research evidence in decision making include limited time4 and skills to find and appraise research evidence,5 unavailability of research when it is required,5 6 21 irrelevance of research5 and presentation in formats that decision makers cannot readily use.21 22 Drawing on theoretical frameworks to promote KT and EIHP,2 23 there are increasing initiatives14 17 being implemented and evaluated to address these barriers keeping in mind that policymaker questions move beyond effects of interventions to questions about implementation and contextual relevance.24
Studies have mapped existing relationships between researchers and decision makers,11 25 and surveyed academics to identify factors influencing engagements between academics and public health decision makers.26 Many have called for initiatives to build relationships,27 some have tried mentorship28 but few have implemented and evaluated strategies to build relationships between policymakers and researchers.
We found that building relationships and dialogue took time and required flexibility and commitment from both sides. Our buddying programme built and strengthened relationships between policymakers and researchers, helped researchers to work with policymakers and to learn how they might be helpful. Both parties recognised that the structured buddying made the dialogue happen. The programme was implemented at subnational level, which also meant that many policy frameworks had already been developed at the national level for provincial policymakers to adapt them, thus reducing incentives (and opportunities) for true instrumental use20 of evidence. Often the evidence was helpful in supporting provincial policy decisions that were in the roll-out phase from the national government.
We would encourage small-scale projects that set parameters for structured engagements such as this, as they help researchers to understand the information requirements of policymakers, enhance dialogue and build relationships that ultimately benefit both groups.
Acknowledgments
The Policy BUDDIES project was a collaborative project by Centre for Evidence-based Health Care (www.sun.ac.za/cebhc); Health Systems and Services Research Unit, Stellenbosch University; Cochrane South Africa (www.mrc.ac.za/cochrane/cochrane.htm); Liverpool School of Tropical Medicine (www.evidence4health.org) and Centre for the Development of Best Practices in Health, Cameroon. We acknowledge all the researchers and policymakers.
References
Footnotes
Handling editor Valery Ridde
Contributors TY: led the conceptualisation and implementation of the Policy BUDDIES project. CN, TK, CW, PG: were part of the project implementation. JS: conducted the independent evaluation of the project. TY: drafted the manuscript and all authors provided input. All authors have approved the manuscript.
Funding This project was supported by the Alliance for Health Policy and System Research, World Health Organisation. Paul Garner, Taryn Young and Celeste Naude were partly supported by the Effective Health Care Research Consortium funded by UK aid from the UK Government for the benefit of developing countries (grant: 5242).
Disclaimer The views expressed in this publication do not necessarily reflect UK government policy.
Competing interests Taryn Young, Celeste Naude, Tamara Kredo, Charles Wiysonge and Paul Garner are identified with Cochrane. All receive grant money from public bodies based on activities related to producing systematic reviews and promoting their uptake.
Ethics approval Stellenbosch University Health Research Ethics Committee provided ethical approval (N13/02/021). All participants gave written informed consent.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement The datasets generated and/or analysed during the current study are available from the corresponding author on reasonable request.