You are here

Article Text

Article menu

Download PDFPDF

Research
Primary health care financing interventions: a systematic review and stakeholder-driven research agenda for the Asia-Pacific region
  1. Blake Angell1,
  2. Rebecca Dodd1,
  3. Anna Palagyi1,2,
  4. Thomas Gadsden1,
  5. Seye Abimbola1,2,
  6. Shankar Prinja3,
  7. Stephen Jan1,
  8. David Peiris1
  1. 1 The George Institute for Global Health, University of New South Wales, Sydney, New South Wales, Australia
  2. 2 School of Public Health, Faculty of Medicine and Health, Sydney University, Sydney, New South Wales, Australia
  3. 3 School of Public Health, Post Graduate Institute of Medical Education and Research, Chandigarh, India
  1. Correspondence to Dr Blake Angell; bangell{at}georgeinstitute.org.au

Abstract

Introduction Interventions targeting the financing of primary health care (PHC) systems could accelerate progress towards universal health coverage; however, there is limited evidence to guide best-practice implementation of these interventions. This study aimed to generate a stakeholder-led research agenda in the area of PHC financing interventions in the Asia-Pacific region.

Methods We adopted a two-stage process: (1) a systematic review of financing interventions targeting PHC service delivery in the Asia-Pacific region was conducted to develop an evidence gap map and (2) an electronic-Delphi (e-Delphi) exercise with key national PHC stakeholders was undertaken to prioritise these evidence needs.

Results Thirty-one peer-reviewed articles (including 10 systematic reviews) and 10 grey literature reports were included in the review. There was limited consistency in results across studies but there was evidence that some interventions (removal of user fees, ownership models of providers and contracting arrangements) could impact PHC service access, efficiency and out-of-pocket cost outcomes. The e-Delphi exercise highlighted the importance of contextual factors and prioritised research in the areas of: (1) interventions to limit out-of-pocket costs; (2) financing models to enhance health system performance and maintain PHC budgets; (3) the design of incentives to promote optimal care without unintended consequences and (4) the comparative effectiveness of different PHC service delivery strategies using local data.

Conclusion The research questions which were deemed most important by stakeholders are not addressed in the literature. There is a need for more research on how financing interventions can be implemented at scale across health systems. Such research needs to be pragmatic and balance academic rigour with practical considerations.

  • primary health care
  • health financing
  • systematic review
  • asia-pacific region
  • health systems
  • research priorities
  • delphi process

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/bmjgh-2019-001481

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Key questions

    What is already known?

    • Effective and sustainable primary health care systems are essential if the low-income and middle-income nations of the Asia-Pacific region are to achieve universal health coverage.

    • Financing interventions have been used across the world to incentivise the demand for and delivery of quality healthcare.

    • The role that research plays in informing policy decisions is often limited by insufficient attention to context, policymakers’ priorities and the broader system-wide impacts of interventions.

    What are the new findings?

    • While national primary health care stakeholders see an important role for financing interventions in the push to achieve universal health coverage in the Asia-Pacific region, there are key gaps in the evidence needed to inform policy decisions.

    • Evidence priorities for primary health care financing include the role of interventions at the: (1) community level, to improve access to services and financial protection of individuals; (2) provider level, to incentivise appropriate care and ensure appropriate management decisions and (3) system level, to improve performance of primary care systems.

    What do the new findings imply?

    • Filling priority evidence gaps will require robust partnerships between researchers and policymakers and the development of pragmatic methods that balance academic rigour with the practical challenges of implementation.

    Background

    Affordable, equitable and high-quality primary health care (PHC) is essential for achieving universal health coverage (UHC). Financing interventions to alter the incentives facing different health system actors, including those working in PHC systems, have been proposed as potential solutions to a range of challenges facing low-income and middle-income countries (LMICs). These challenges include improving the quality and appropriateness of care provided,1–5 reducing wasteful expenditure6 7 and overcoming workforce shortages and the maldistribution of health workers.8 However, evidence supporting the effectiveness of financing interventions remains mixed, and many have pointed to the potential for such interventions to have negative, unintended consequences when scaled-up, highlighting the need for financing interventions to be carefully calibrated to the context in which they are implemented.

    The Asia-Pacific region is home to two-thirds of the global population.9 The LMICs of the region range from the most populous countries of the world in South and East Asia, to the least across the Pacific Island nations. While all countries in the region are pursuing UHC, the policy initiatives being implemented, and the context and challenges of each system vary dramatically. The more populous countries such as India,10 China,11 Vietnam12 and Indonesia13 are embarking on large programmes of reform to overcome historical underinvestment and provide health and financial protection for their citizens. High rates of economic growth in these countries in recent decades have stimulated an increase in the amount that payers (government, industry and consumers) are prepared to invest in health care.14 15 The sustainability of this growth in health care investment, together with how and where such funds are allocated, will be strong determinants of how these nations progress towards UHC. Pacific Island nations face very different challenges: here, health expenditure has been historically high relative to their income levels,15 but service delivery costs are also high due to small, remote, widely dispersed and expanding populations.16–19

    Across all nations of the region, the efficiency of existing health expenditure has been questioned7 and the challenges facing these health systems have been magnified in recent years by the rising burden of non-communicable diseases.20 This has stretched resources and challenged the adaptability of health systems that were historically oriented to episodic, curative care models rather than the long-term care needs of those with, or at risk of, chronic diseases.21 The scope of reforms currently underway across the region provides a unique opportunity to understand optimal paths to UHC across these different contexts. There is limited evidence on how to best finance and incentivise PHC systems to encourage appropriate, cost-effective, equitable, high quality and sustainable care.21 22 Identifying research questions that meet the needs of policymakers looking to accelerate progress towards UHC23 24 is therefore a priority.

    In this paper, we present an iterative process to develop a prioritised agenda to inform future PHC financing research in LMICs of the Asia-Pacific, incorporating the perspectives of researchers, policymakers and other national PHC system stakeholders across the region.

    Methods

    This study forms part of a larger project examining evidence gaps in the organisation of PHC service delivery in LMICs of the Asia-Pacific region.25 26 We followed a two-phase process to identify and prioritise evidence gaps related to PHC financing interventions. Phase 1 was a systematic review of the peer-reviewed and grey literature to identify core knowledge gaps in the published research. Phase 2 involved an electronic-Delphi (e-Delphi) priority-setting exercise with key PHC stakeholders in Asia-Pacific LMICs to refine and prioritise research questions related to PHC financing interventions.

    Patient and public involvement

    No patients or members of the public were involved in this study.

    Phase 1: systematic review

    A systematic review of the peer-reviewed and grey literature was carried out to identify studies of the effectiveness of financing interventions implemented within PHC systems of LMICs in the Asia-Pacific region.

    Inclusion criteria

    The inclusion criteria for the review specified three criteria. First, studies had to evaluate financing interventions in relation to the key PHC outcomes of quality, coverage and accessibility, efficiency, responsiveness or equity.26 Financing interventions were defined broadly to include any intervention that altered the incentives facing PHC providers or patients, changed the ownership structure of PHC services or altered the fees faced by patients. Health economic evaluations that assessed the cost-effectiveness of various PHC interventions were excluded if the interventions did not meet these criteria. Second, studies had to be published between 1 January 2008 and February 2018. Third, studies had to have been conducted in the LMICs of the World Bank regions of South Asia and East Asia and Pacific. Studies examining an intervention implemented in multiple countries were included if country-specific results were reported or if all included nations were located in these regions.

    Search strategy

    A search of the Medline, Global Health and Cochrane databases was conducted in February 2018 using variations of the search terms contained in online supplementary file 1A. We also hand-searched grey literature from 14 websites of organisations and agencies from the international development and global health sector. Reference lists of the final included publications were hand-searched for additional articles relevant to the review, and further relevant articles were identified through expert consultation.

    Supplemental material

    Data extraction

    Study review, selection and data extraction were jointly undertaken by two authors (AP and RD). Abstracts, titles and keywords of the studies returned from the search were screened for compatibility with the inclusion criteria. Once studies were identified for potential inclusion, full texts were reviewed to determine inclusion into the review. Due to the likely heterogeneity of the methods of included studies and the underlying purpose of the review to inform the e-Delphi phase of the study, no formal quality assessment was carried out of reviewed studies, nor were there any quality criteria for inclusion or exclusion. The full text of each included publication was reviewed to identify key study characteristics (context, geographical scope, research method, study population, disease focus) and the PHC system outcomes and inputs addressed. Each of these elements was populated in a predefined coding framework using EPPI-Reviewer 4 (EPPI-Centre, Social Science Research Unit, Institute of Education, University College London, UK) to generate an ‘evidence gap map’. Studies were grouped according to the following themes: resource allocation and budgeting; revenue collection; funding models; provider incentives and demand-side incentives. These were mapped against PHC outcomes of quality of care, efficiency, coverage and accessibility, responsiveness and equity.

    Phase 2: modified e-Delphi expert panel

    Following the systematic review, an e-Delphi expert panel was convened over April–June 2018 using a web-based data collection tool (Research Electronic Data Capture). Stakeholders were identified through the authors’ existing networks and snowball sampling. Despite the authors’ strong network of China stakeholders, we discovered prior to the initiation of the Delphi exercise that firewall restrictions precluded use of the online survey with China-based participants so no stakeholders from China were included in this study. In total, 179 email invitations to join the panel were sent to stakeholders from 15 Asia-Pacific LMICs and 15 regional bodies. While we endeavoured to achieve a diverse sample of professions in the e-Delphi process, the panel was based on the authors’ prior networks and, as such, we were unable to make further efforts to balance the countries and professions represented. The characteristics of the panel are summarised in table 1. This process was part of a larger study that also examined the organisation of PHC.25

    View this table:
    Table 1

    Country of work and professional background of the 22 Delphi expert panel members

    The e-Delphi was an iterative prioritisation process involving three rounds of web-based surveys with the expert panel members. Panel members were asked to complete each Delphi round within 10 days and automated email reminders were sent to each prior to each round closing.

    Round 1: topic suggestions

    The first round of the e-Delphi process was based on the findings of the systematic review and evidence gap analysis. Participants were presented with a list of broad research gaps identified by the mapping exercise and asked to assign a priority score of 1–5, for each topic (where 1 is the lowest research priority and 5 the highest). Participants were also asked to identify specific research questions of interest that fit within their top two priority topics.

    Round 2: prioritisation

    The three highest scoring research topics from Round 1 were identified and the research questions generated thematically grouped within these. The questions were then fed back to panel members who were asked to prioritise the research questions according to the priority setting criteria established by the primary health care performance initiative (outlined in box 1).27 These criteria aim to prioritise research most likely to impact on PHC system performance.

    Box 1

    Priority setting criteria established by the primary health care performance initiative

    Priority tiers:

    • Tier 1: includes research gaps of high potential leverage, in which more evidence is likely to have a significant impact on strengthening PHC performance.

    • Tier 2: includes research gaps of moderate potential leverage, in which more evidence is somewhat likely to have an impact on strengthening PHC performance.

    • Tier 3: includes research gaps for which more evidence is unlikely to have an impact on strengthening PHC performance.

    Priority setting criteria:

    • Available knowledge: What is the current level and quality of available knowledge? (Areas with lower available knowledge should be prioritised for further exploration.)

    • Leverage: What is the potential for improvements in this area to contribute to strengthening PHC performance? (Areas that are seen as key levers of improvement should be prioritised.)

    • Magnitude of need: How crosscutting is this problem in your setting?

    • Equity: How likely is research on improving the areas also likely to reduce disparities

    • Innovation: How likely is the research to stimulate innovation in how PHC is strengthened or innovation in how known interventions are implemented?

    • Can research address the gap?

    Round 3: final selection of priority topics

    Research questions that were ranked under the top two priority tiers were fed back to panel members, who were requested to assign each question a priority score of 1–5, as in Round 1. Responses of panel members were then collated and an overall median priority score (1–5) was calculated for each research question. Research gaps with a median priority score of 4 or 5 (ie, the two highest priority levels) were ranked to present a ‘top 10’ list of priority questions.

    Results

    Phase 1: systematic review

    The initial search yielded 3001 articles. Once duplicates were removed, the titles and abstracts of 1919 articles were screened plus 14 more found through reference list searches and expert advice. In total, 155 full texts were reviewed and 31 met the inclusion criteria for this review (figure 1).28–58 In addition, the search returned 10 systematic reviews of relevance to our study.1 8 59–66

    Figure 1
    Figure 1

    Flow chart of literature search and inclusion. LMIC, low-income and middle-income countries; PHC, primary health care.

    Study characteristics

    Twenty-one original research peer-reviewed publications and 10 grey literature reports were included in the review. A summary of these studies is presented in the online supplementary tables S1 and S2 and an overview of the systematic reviews found through our search is provided in supplementary table S3. Reflecting the broad nature of our inclusion criteria, studies examined a diverse set of financing interventions, across a number of nations and included evaluations using a range of methodologies. Included original research studies were conducted across a total of eight LMICs of the Asia-Pacific region: China (nine articles), Vietnam (three articles), Nepal (two articles), Afghanistan (two articles) and Pakistan, India, Bangladesh, American Samoa and Timor-Leste (one each). One grey literature study covered three nations (China, Indonesia and the Philippines); there were two grey literature reports from Indonesia, two from India and one each from China, the Philippines, Vietnam, Papua New Guinea and Lao PDR. No studies were found that linked financing interventions to improved health outcomes. Studies were coded against the predefined input and outcome categories as described above to identify gaps in the literature (table 2) and are examined below based on the type of PHC financing intervention studied.

    View this table:
    Table 2

    Evidence gap map showing primary health care financing studies in the Asia-Pacific region examining selected issues

    Pay-for-performance interventions

    Five peer-reviewed studies and one grey literature report investigated pay-for-performance interventions that aimed to incentivise health workers by paying for agreed indicators of service quality.34 42 43 47 48 58 The specific design of each intervention differed across the studies. Engineer and colleagues examined the impact of a pay-for-performance scheme on maternal and child health services in Afghanistan using a cluster randomised controlled trial design.34 They found no impact on coverage and equity measures but some impact on indicators of quality of care. The authors conclude that overall the programme had limited impact and attribute this to communication difficulties between policymakers and health workers and failure to consider demand-side factors. Two studies investigated pay-for-performance schemes in conjunction with capitation payment structures in China.43 47 Sun used an interrupted time-series design and concluded that the combination of capitation and pay-for-performance (based on financial performance of the service) led to an increase in primary care visits, a reduction in inpatient admissions and a decreased cost of outpatient visits. However, performance payments had no impact on reducing inappropriate prescribing practices which was a key aim of the programme.47 Powell-Jackson and colleagues attempted to separate out the impact of the pay-for-performance intervention from the capitation intervention also using an interrupted time series conducted across two policy changes. They found that performance payments had no additional impact on health service utilisation above that associated with the introduction of an insurance programme.43 A World Bank study found paying doctors for appropriate prescribing had a positive effect, reporting a significant reduction in inappropriate prescribing using a cluster quasi-randomised controlled study.48 Finally, Powell-Jackson and colleagues investigated the provision of incentives to both service providers and mothers to encourage use of skilled birth attendants.42 Using an interrupted time series design, the authors found an improvement in skilled birth attendance, but only in areas where women’s groups disseminated information about the programme. There was no impact on neonatal mortality across any site.

    Removal of user fees and public insurance

    Seven peer reviewed studies and four grey literature reports examined the impact of removing user fees for care or the provision of publicly funded insurance (on top of the interventions combined with pay-for-performance described above).28 30 32 33 39 41 45 49 52 53 57 The context and form of these interventions differed greatly across the included studies; however, there were relatively consistent results that these schemes reduced costs for users and increased PHC utilisation. Two studies from China showed that removing fees (while also compensating providers by increasing their reimbursement) led to significant increases in the use of outpatient care.43 47 Similarly, Thanh and colleagues conducted an assessment of the impact of removing user fees in Vietnam through propensity-matched cohort study using household panel data and also found that the removal of user fees for the poor led to an increased utilisation of PHC services and reduced out-of-pocket health care expenditure.52

    The other included studies, however, reported challenges in implementing or expanding similar interventions that often diluted their overall impact. A study in Nepal examining the removal of user fees using paired case studies and qualitative research found beneficial, pro-poor impacts on utilisation in some facilities but no impact in others. The authors argued that this was a result of weak organisational capacity (in some facilities) to implement the reform.45 Nguyen found that a poverty reduction policy that made basic curative and preventative health services free of charge did not impact healthcare utilisation in remote regions of Vietnam.41 Rather, even for the poor, uptake was dependent on the perceived quality of care regardless of cost. Similarly, four World Bank studies examined the impact of expanded insurance programme and all found contextual factors had a large impact on their effectiveness. In Indonesia, a pre–post study found that Jampersal, an insurance programme to cover maternal services, was effective in reaching women particularly those who were least educated, poor and resided in rural areas.28 Barroy and colleagues found that while the Vietnamese national insurance programme had reached relatively high coverage (60%) and there had been accompanying quality and equity improvements, out-of-pocket expenditures remained high and the programme was limited by ineffective risk-pooling and capitation.30

    Weak governance arrangements between central and provincial governments in Papua New Guinea were found to limit frontline PHC service provision in the expenditure analysis conducted by Cairns and Hou.33 While noting that public programmes had increased coverage across the Philippines, Bredenkamp and Buisman highlighted the challenges that remained with regard to the financial protection of the population, noting that the poor still experienced significantly higher levels of catastrophic expenditure.32 Finally, Mahmood highlighted shortcomings in a social insurance scheme in rural China, finding that the scheme’s gap payment acted as a significant barrier to care.39

    Contracting out PHC services

    Six studies assessed the impact of contracting out PHC service provision to private and non-government providers or alterations to the ownership structure of PHC services.29 36–38 55 56 A number of these found positive impacts in terms of coverage, efficiency and equity outcomes. In Afghanistan, Alonge and colleagues found that contracts that allowed providers freedom to allocate a fixed lump sum payment, combined with non-negotiable deliverables, were effective in improving equity of health services provision.29 In Bangladesh, Heard and colleagues found that contracting out urban PHC services to a non-government organisation (NGO) led to improvements in the coverage of basic services, equity, quality of care and efficiency.36 In Pakistan, it was found that contracting out PHC services in one poor and rural district to a local NGO resulted in increased coverage, community satisfaction and efficiency.38

    Potential negative impacts of contracting services were related to quality control, highlighted by two studies in China comparing different models of ownership of healthcare centres. One study found that private providers had the lowest quality of care and the least educated staff.54 Similarly, another study found that government and hospital managed centres provided better quality care in terms of appropriate prescribing practices and optimal blood pressure control for patients with hypertension.61 A report by the WHO found that expanding social health insurance via outsourcing services can impact negatively on equity outcomes, especially when schemes are rapidly scaled up.37

    Incentives for patients

    Three peer reviewed studies and two grey literature reports examined the impact of incentives for to encourage patients’ use of PHC services.35 40 42 46 53 Once again contextual factors were often found to play a vital role in the impact of such programmes. As highlighted above, patient incentives (as well as provider incentives) were only effective in improving skilled birth attendance in places with women’s groups, suggesting social networks are critical to the success of such programmes. In addition, the authors found that benefits accrued disproportionately to wealthier households. In American Samoa, a slight improvement in the attendance of adequate prenatal care services was found following the introduction of an incentive programme for expectant mothers; however, these effects were not able to overcome system limitations that made care inaccessible for many women in the study.35

    Using a randomised controlled trial, Martins and colleagues found limited impact in terms of adherence or the outcome of treatment from providing food incentives to patients with tuberculosis in East Timor.40 By contrast, a study in India found the greatest reduction in inequities from social insurance occurred in states with a concurrent programme of conditional cash transfers to encourage attendance at maternal health services.53 Similarly, a study in Lao PDR examined the impact of a community nutrition programme, comprising community education and conditional cash transfers, to determine impact on health-seeking behaviours among mothers who were pregnant or had a child under 2 years.51 The intervention was found to improve the rate of diphtheria, pertussis and tetanus vaccinations and child-caring practices such as breastfeeding within 3 hours of birth and treatment-seeking for diarrhoea. Finally, an Indonesian study found that child healthcards (which entitled them to a price subsidy) had a protective effect during the Asian financial crisis, with public outpatient service utilisation declining less steeply compared with non-subsidised services.46

    Key evidence gaps

    The systematic review and mapping exercise identified a number of key gaps in the existing literature. Notably, there were very few studies from Pacific Island countries and, conversely, much of the literature was dominated by studies from China. No studies examined the impact of financing interventions on health outcomes and there was limited work examining the efficiency and responsiveness of PHC systems, pointing to the need for further research on the appropriate role that health system incentives can play in these areas. This reflects the findings of reviews in similar areas across other LMICs as summarised in the online supplementary table S3. Further, the generalisability of the literature that was found for this review was hampered by the diversity of interventions studied, health systems and methodologies used. One of the most consistent results of included studies was the importance of contextual factors that were found to impede or otherwise affect implementation or scale-up of these interventions and subsequently impact on the outcomes of policy interventions. These identified evidence gaps were used to formulate five broad areas for potential further research, which were presented to the Delphi panel in the first round of the e-Delphi process, as described below:

    Supplemental material

    1. Mobilising resources to pay for PHC.

    2. Contracting models to ‘purchase’ PHC.

    3. Optimising efficiency and cost-effectiveness of PHC.

    4. Financial incentives to improve demand for PHC services.

    5. Financial management at PHC level.

    Phase 2: e-Delphi results

    In Round 1 of the e-Delphi exercise, 22 panel members (n=22) ranked the five topics above and identified 34 related research questions relevant to PHC financing in the countries in which they worked. Following the removal of duplicate questions and the thematic analysis outlined above, a final set of 18 research questions were derived for presentation in Round 2. Fourteen of the original 22 expert panel members completed Round 2 of the e-Delphi exercise. Non-respondents included panel members from Fiji and Bangladesh (one from each country), two panel members representing regional programmes of the aid/development sector and two stakeholders from India (from academic and aid/development sectors). Of these 18 research questions, 14 received an aggregate ranking within the top two priority tiers (ie, Tiers 1 and 2) and were collated for final prioritisation in Round 3.

    Fourteen expert panel members completed Round 3. Eight questions received a median priority score of 4 or 5 (ie, two highest priority levels). We aimed to achieve a minimum of 80% agreement for those research questions of highest priority; however, we achieved 60% for six of the top questions only. Reasons for this might include the small panel size (only 14 of 22 (64%) original panel members completed Rounds 2 and 3 of the e-Delphi exercise), evidence priorities differing between countries within the region and the less obvious link between some of the evidence gaps and implementation research (eg, use of essential medicines and technologies to improve responsiveness).

    The final list of priority questions are presented in table 3. They represent an array of evidence needs that overlap in some areas with those identified through the literature but differ in others. Research questions focussed on interventions targeting different actors in the system—the community, service providers and policymakers themselves. The top-ranked question identified by the panel was: how can the poor be protected from impoverishing payments for catastrophic health expenditure, including the purchase of medicines? Interest coalesced around a set of questions related to efficiency and cost-effectiveness, with five of the top 10 research questions prioritised by this panel relating to these issues. Another key area of interest prioritised by the e-Delphi panel related to mobilising resources for PHC.

    View this table:
    Table 3

    Consensus on Asia-Pacific research priorities in financing of primary health care, in rank order

    Notably, the financial management of PHC resources was identified as a key knowledge gap, with research questions on the organisational capacity of health centres to manage PHC finances, and the optimal use of financial resources emerging as priorities. This issue is a critical element of PHC governance and relates to the capacity of health systems to manage, distribute and account for financial resources. No peer-reviewed study (and only one piece of grey literature) was identified on this issue, suggesting a major gap in evidence.

    Discussion

    This evidence review and stakeholder consultation process identified the need for more research on the optimal design(s) of PHC financing interventions at the system, provider or community level in LMICS of the Asia-Pacific region. We also identified the need for research into how to best implement these interventions and the critical need to understand contextual constraints and opportunities.

    In order to be of use to policymakers in the UHC era, health systems research needs to answer questions that matter for population health, offer contextually appropriate results in a form that can be used by policymakers and be methodologically innovative to balance rigorous assessment of causality of outcomes with practical considerations of policy implementation.23 This study suggests that the current evidence base may not meet these requirements. There is a need for more research across a broader number of LMICs. Developmental and impact evaluations are needed that report on service utilisation, efficiency, provider and patient behaviour, financial risk protection and the health outcomes of populations. Where feasible, comparative effectiveness studies that examine the relative impacts of different financing models on different outcomes are likely to be of interest to policymakers.

    To help address these issues, we have proposed a list of priorities for research to drive progress towards UHC in the Asia-Pacific region. Although there were studies that partially address some of the gaps raised by the e-Delphi participants (eg, patient incentives to improve PHC utilisation), most of the questions deemed most important by stakeholders could not be adequately addressed by existing evidence and did not align with the gaps we initially identified. At the community level, the top priority area was the need for more research around models that promote financial protection for the poor. While a growing body of global literature has documented the economic burden and impoverishment associated with healthcare costs,67–69 the literature examining interventions to overcome these is not well developed. While there was existing evidence that removing or reducing user fees or expansion of publicly funded services increased PHC utilisation, there was limited focus on specific interventions to limit out-of-pocket payments, catastrophic health expenditure and medical-related impoverishment. Further, there was variation across the results of included studies reflecting the complexity of interventions, the context in which they were implemented and the scale at which they were implemented.

    Across included studies, interventions often resulted in diluted impacts when rapidly scaled, casting more doubt on the usefulness of such research for policymakers and highlighting the need for pragmatic methods to assess the impact of financing interventions. Studies using methods such as natural experiments in a wide range of countries and contexts, at different scales of implementation and altering different levers in the design of financing interventions will be vital to tease out the best strategies for protecting populations from financial stress.

    Stakeholders highlighted a range of provider-level research questions around factors such as managing budgets and identifying inefficient expenditure that were not considered in the studies included in the review. Provider-level issues such as how to best incentivise quality services (both financially and non-financially) were the second major category for future research highlighted in this study. There are opportunities for considerable innovation in both developing these incentives and the research methodologies needed to assess them.70–72 Consideration of the trade-offs within health systems, for example, between patient choice, quality of care, efficiency and the relative investment in primary as opposed to other levels of care, and how to appropriately incentivise provider behaviours in light of these needs to form the basis for future research in the area. To be applicable to the different contexts of the LMICs of the Asia-Pacific region, such work needs to account for local preferences and community values, again opening up avenues for methodological innovation.

    Research on system-wide policy reforms were also identified as a priority area. Interaction between primary and other levels of the healthcare systems was consistently highlighted as an issue for financing reform. Ensuring that resources could be directed to primary care and not diverted to latter stage care, using incentives to encourage patients to make greater use of primary care services, and removing barriers to care were key priorities to emerge from this study. Traditional ‘gold-standard’ approaches such as randomised controlled trials may be unsuitable to assess the impact of such interventions. Included studies show that while all health systems are complex, they are complex in different ways, and this context determines the impacts of interventions and the likely success of scale-up. This suggests the need for research and evaluation methods that address the channels through which financing interventions impact on actor behaviour and health system performance. Some promising developments have been identified in the existing literature based on methods such as difference in difference, time series analyses and propensity score matching often using routinely collected data.73 74 Other pragmatic designs such as stepped-wedge randomised controlled trials may be needed to tease out the impacts of system-wide interventions across complex health systems.75

    There were some limitations to this study. Our focus was PHC; it is possible that in some studies PHC is described in different ways to the search strategy laid out in the online supplementary appendix A, which may have led to them not being included in our review. Our search was limited to English language studies, which may again miss some work that has been done in the field. Further, our search of the grey literature focused on specific preidentified websites and as such was not comprehensive. We were limited by the small Delphi panel size (only 22/179 (13%) invited participants agreed to join and 14/22 (64%) original panel members completed rounds 2 and 3). While the initial response rate is reflective of response rates previously achieved by web-based Delphi research,76 it is a notable drawback of this virtual method of engagement that needs to be considered alongside benefits of efficiency and geographical reach. The panel did not reach a clear consensus on the research priorities and we were unable to analyse regional variation in priority evidence needs. Because of the limited panel sample size, we urge caution in interpreting the degree of concordance between the literature-informed evidence gaps and stakeholder research priorities. The exclusion of China stakeholders, in particular, was a limitation given the predominance of research evidence from China PHC systems. Alternative approaches to engaging China PHC stakeholders in priority setting activities, including roundtable discussions or the face-to-face nominal group technique, are recommended in future. Lack of expert panel representation from Pacific Island countries in the latter stages of the Delphi process also restricts the geographic relevance of identified research priorities. Further work involving both deeper engagement with panel members (perhaps through in-depth interviews) and broader engagement with a larger number of stakeholders from across the region could corroborate the findings presented here.

    In addition, the inability to include China stakeholders, in particular, represents a significant limitation given the predominance of evidence from China PHC systems. Alternative approaches to engaging China PHC stakeholders in priority setting activities, including roundtable discussions or the face-to-face nominal group technique, are recommended in future. Lack of expert panel representation from Pacific Island countries in the latter stages of the Delphi process also weakens the geographic relevance of identified research priorities. Clearly, the findings would need to be corroborated with both deeper engagement with panel members (perhaps through in-depth interviews) and broader engagement with a larger number of stakeholders from across the region. Future studies may wish to use this work to develop similar prioritised research agendas at subregional, national or subnational level.

    Conclusion

    The impact of research on policy implementation and progress towards global targets such as UHC is too often limited by a lack of consideration of policymaker’s perspectives and local context. By involving policymakers and other key stakeholders, we have produced a prioritised research agenda that provides insights into the key needs of policymakers as they implement policies pursuant to UHC. The key priorities to emerge covered three levels of intervention: (1) community-level interventions to promote access and financial protection; (2) service-level financing models and (3) system-level funding policies. More research is needed which examines PHC systems and providers through an economic lens and there is substantial room for methodological innovation in health systems research across the Asia-Pacific region.

    Supplemental material

    Supplemental material

    Acknowledgments

    Preliminary results of this work were presented at the 5th Global Symposium on Health Systems Research in Liverpool in October 2018 at a satellite session entitled ‘Understanding and addressing knowledge gaps in improving primary health care systems and delivery in LMICs from governance through quality: findings from an emerging research consortium’.

    References

    1. 1.↵
      2. Oyo-Ita A ,
      3. Wiysonge CS ,
      4. Oringanje C , et al
      . Interventions for improving coverage of childhood immunisation in low- and middle-income countries. Cochrane Database Syst Rev 2016;7.doi:10.1002/14651858.CD008145.pub3
    2. 2.↵
      2. Oxman AD ,
      3. Fretheim A
      . Can paying for results help to achieve the millennium development goals? A critical review of selected evaluations of results-based financing. J Evid Based Med 2009;2:184–95.doi:10.1111/j.1756-5391.2009.01024.x
      OpenUrlPubMed
    3. 3.↵
      2. Savedoff WD
      . Incentive proliferation? making sense of a new wave of development programs 2011.
    4. 4.↵
      2. Mills A
      . Health care systems in low- and middle-income countries. N Engl J Med Overseas Ed 2014;370:552–7.doi:10.1056/NEJMra1110897
      OpenUrl
    5. 5.↵
      2. Kok MC ,
      3. Dieleman M ,
      4. Taegtmeyer M , et al
      . Which intervention design factors influence performance of community health workers in low- and middle-income countries? A systematic review. Health Policy Plan 2015;30:1207–27.doi:10.1093/heapol/czu126
      OpenUrlCrossRefPubMed
    6. 6.↵
      2. Turcotte-Tremblay A-M ,
      3. Spagnolo J ,
      4. De Allegri M , et al
      . Does performance-based financing increase value for money in low- and middle- income countries? A systematic review. Health Econ Rev 2016;6.doi:10.1186/s13561-016-0103-9
    7. 7.↵
      1. World Health Organization
      . World Health Report, 2010: health systems financing the path to universal coverage 2010.
    8. 8.↵
      2. Grobler L ,
      3. Marais BJ ,
      4. Mabunda S
      . Interventions for increasing the proportion of health professionals practising in rural and other underserved areas. Cochrane Database Syst Rev 2015;(6).doi:10.1002/14651858.CD005314.pub3
    9. 9.↵
      1. The United Nations
      . Asia and the Pacific, 2018. Available: http://www.un.org/en/sections/where-we-work/asia-and-pacific/ [Accessed 12 Nov 2018].
    10. 10.↵
      2. Bhargava B ,
      3. Paul VK
      . Informing ncd control efforts in India on the eve of Ayushman Bharat. The Lancet 2018.doi:10.1016/S0140-6736(18)32172-X
    11. 11.↵
      2. L.J L
      . China through the lens of health in 2018 and beyond 2018;391:999.
    12. 12.↵
      2. Reich MR ,
      3. Harris J ,
      4. Ikegami N , et al
      . Moving towards universal health coverage: lessons from 11 country studies. The Lancet 2016;387:811–6.doi:10.1016/S0140-6736(15)60002-2
      OpenUrl
    13. 13.↵
      2. Mboi N ,
      3. Murty Surbakti I ,
      4. Trihandini I , et al
      . On the road to universal health care in Indonesia, 1990–2016: a systematic analysis for the global burden of disease study 2016. Lancet 2018;392:581–91.doi:10.1016/S0140-6736(18)30595-6
      OpenUrl
    14. 14.↵
      2. Gerdtham U-G ,
      3. Jönsson B
      . Chapter 1 – international comparisons of health expenditure: theory, data and econometric analysis. In: Chalkley M , Culyer T , Doorslaer Evan , et al. , eds. Handbook of health economics. Elsevier, 2000: 11–53.
    15. 15.↵
      2. Dieleman J ,
      3. Campbell M ,
      4. Chapin A , et al
      . Evolution and patterns of global health financing 1995–2014: development assistance for health, and government, prepaid private, and out-of-pocket health spending in 184 countries. The Lancet 2017;389:1981–2004.doi:10.1016/S0140-6736(17)30874-7
      OpenUrl
    16. 16.↵
      2. Peter Richard Wallace ATC ,
      3. Archambault MR ,
      4. Ivatts SL
      . Vanuatu – health financing systems assessment: spend better. The World Bank Group, 2018.
    17. 17.↵
      2. Flanagan RJC ,
      3. Thomas A ,
      4. Ruest M , et al
      . Kiribati - Health financing system assessment: spend better 2018.
    18. 18.↵
      2. Hou XJ ,
      3. Brown Laurin ,
      4. Turner SV
      . Health financing system assessment: Papua New Guinea. World Bank Group, 2018.
    19. 19.↵
      2. Ruest MI ,
      3. Lynette S ,
      4. Flanagan B
      . Solomon Islands – health financing system assessment: spend better. World Bank Group, 2018.
    20. 20.↵
      2. Samb B ,
      3. Desai N ,
      4. Nishtar S , et al
      . Prevention and management of chronic disease: a litmus test for health-systems strengthening in low-income and middle-income countries. The Lancet 2010;376:1785–97.doi:10.1016/S0140-6736(10)61353-0
      OpenUrl
    21. 21.↵
      2. Dans A ,
      3. Ng N ,
      4. Varghese C , et al
      . The rise of chronic non-communicable diseases in Southeast Asia: time for action. The Lancet 2011;377:680–9.doi:10.1016/S0140-6736(10)61506-1
      OpenUrl
    22. 22.↵
      1. WHO Western Pacific Region
      . Healthy Islands, in twelfth Pacific health ministers meeting. Rarotonga, Cook Islands: WHO Western Pacific Region, 2017.
    23. 23.↵
      2. Leslie HH ,
      3. Hirschhorn LR ,
      4. Marchant T , et al
      . Health systems thinking: a new generation of research to improve healthcare quality. PLoS Med 2018;15:e1002682.doi:10.1371/journal.pmed.1002682
    24. 24.↵
      2. Sheikh K ,
      3. George A ,
      4. Gilson L
      . People-centred science: strengthening the practice of health policy and systems research. Health Res Policy Syst 2014;12:19.doi:10.1186/1478-4505-12-19
    25. 25.↵
      2. Palagyi A ,
      3. Dodd R ,
      4. Jan S , et al
      . Organisation of primary health care in the Asia-Pacific region: developing a prioritised research agenda. BMJ Global Health 2019;4:e001467.doi:http://​dx.​doi.​org/​10.​1136/​bmjgh-​2019-​001467
    26. 26.↵
      2. Dodd R ,
      3. Palagyi A ,
      4. Jan S
      . Organisation of primary health care systems in low- and middle-income countries: review of evidence on what works and why in the Asia-Pacific region. BMJ Global Health 2019;4:e001487.doi:http://​dx.​doi.​org/​10.​1136/​bmjgh-​2019-​001487
    27. 27.↵
      1. Consortium P.H.C.M.a.I.R
      . Priority setting meeting report. Boston, 2017.
    28. 28.↵
      2. Achadi ELA ,
      3. Pambudi Anhari ,
      4. Marzoeki Eko
      . A study on the implementation of Jampersal policy in Indonesia., in health, nutrition, and population (HNP) discussion paper. Washington, DC: World Bank Group, 2014.
    29. 29.↵
      2. Alonge O ,
      3. Gupta S ,
      4. Engineer C , et al
      . Assessing the pro-poor effect of different contracting schemes for health services on health facilities in rural Afghanistan. Health Policy Plan 2015;30:1229–42.doi:10.1093/heapol/czu127
      OpenUrlCrossRefPubMed
    30. 30.↵
      2. Barroy HJ ,
      3. Bales S
      . Vietnam: learning from smart reforms on the road to universal health coverage in health, nutrition, and population (HNP) discussion paper. Washington, DC: World Bank, 2014.
    31. 31.↵
      2. Berman P
      . Government health financing in India: challenges in achieving ambitious goals 2010.
    32. 32.↵
      2. Bredenkamp CB ,
      3. Leander R
      . Policy research working paper, universal health coverage in the Philippines: progress on financial protection goals. Washington, DC: World Bank, 2015.
    33. 33.↵
      2. Cairns AH
      . Financing the Frontline in Papua New Guinea: an analytical review of provincial administrations' Rural Health Expenditure 2006-2012, in Health, Nutrition and Population Discussion Paper. Washington, DC: World Bank, 2015.
    34. 34.↵
      2. Engineer CY ,
      3. Dale E ,
      4. Agarwal A , et al
      . Effectiveness of a pay-for-performance intervention to improve maternal and child health services in Afghanistan: a cluster-randomized trial. Int J Epidemiol 2016;45:451–9.doi:10.1093/ije/dyv362
      OpenUrlCrossRefPubMed
    35. 35.↵
      2. Hawley NL ,
      3. Brown C ,
      4. Nu'usolia O , et al
      . Barriers to adequate prenatal care utilization in American Samoa. Matern Child Health J 2014;18:2284–92.doi:10.1007/s10995-013-1368-9
      OpenUrlCrossRefPubMed
    36. 36.↵
      2. Heard A ,
      3. Nath DK ,
      4. Loevinsohn B
      . Contracting urban primary healthcare services in Bangladesh – effect on use, efficiency, equity and quality of care. Trop Med Int Health 2013;18:861–70.doi:10.1111/tmi.12113
      OpenUrl
    37. 37.↵
      2. Honda A ,
      3. McIntyre D ,
      4. Hanson K
      . Tangcharoensathien Viroj, strategic purchasing in China, Indonesia and the Philippines. Manila, 2016.
    38. 38.↵
      2. Loevinsohn B ,
      3. Haq IU ,
      4. Couffinhal A , et al
      . Contracting-in management to strengthen publicly financed primary health services--the experience of Punjab, Pakistan. Health Policy 2009;91:17–23.doi:10.1016/j.healthpol.2008.10.015
      OpenUrlCrossRefPubMedWeb of Science
    39. 39.↵
      2. Mahmood MA ,
      3. Raulli A ,
      4. Yan W , et al
      . Cooperative medical insurance and the cost of care in Shandong, PR China: perspectives of patients and community members. Asia Pac J Public Health 2015;27:NP897–902.doi:10.1177/1010539510376664
      OpenUrlCrossRefPubMed
    40. 40.↵
      2. Martins N ,
      3. Morris P ,
      4. Kelly PM
      . Food incentives to improve completion of tuberculosis treatment: randomised controlled trial in DILI, Timor-Leste. BMJ 2009;339:b4248.doi:10.1136/bmj.b4248
    41. 41.↵
      2. Nguyen P ,
      3. Bich Hanh D ,
      4. Lavergne MR , et al
      . The effect of a poverty reduction policy and service quality Standards on commune-level primary health care utilization in Thai Nguyen Province, Vietnam. Health Policy Plan 2010;25:262–71.doi:10.1093/heapol/czp064
      OpenUrlCrossRefPubMed
    42. 42.↵
      2. Powell-Jackson T ,
      3. Neupane BD ,
      4. Tiwari S , et al
      . The impact of Nepal's national incentive programme to promote safe delivery in the district of Makwanpur. Adv Health Econ Health Serv Res 2009;21:221–49.
      OpenUrlPubMed
    43. 43.↵
      2. Powell-Jackson T ,
      3. Yip WC-M ,
      4. Han W
      . Realigning demand and supply side incentives to improve primary health care seeking in rural China. Health Econ 2015;24:755–72.doi:10.1002/hec.3060
      OpenUrl
    44. 44.↵
      2. Rao MG ,
      3. Choudhury M
      . Health care financing reforms in India. 2012: National Institute of Public Finance and Policy, 2012.
    45. 45.↵
      2. Sato M ,
      3. Gilson L
      . Exploring health facilities' experiences in implementing the free health-care policy (FHCP) in Nepal: how did organizational factors influence the implementation of the user-fee abolition policy? Health Policy Plan 2015;30:1272–88.doi:10.1093/heapol/czu136
      OpenUrlCrossRefPubMed
    46. 46.↵
      2. Somanathan A
      . The impact of price subsidies on child health care use: evaluation of the Indonesian Healthcard, in policy research working paper 2008. Washington, DC: World Bank, 2008.
    47. 47.↵
      2. Sun J ,
      3. Zhang X ,
      4. Zhang Z , et al
      . Impacts of a new insurance benefit with capitated provider payment on healthcare utilization, expenditure and quality of medication prescribing in China. Trop Med Int Health 2016;21:263–74.doi:10.1111/tmi.12636
      OpenUrl
    48. 48.↵
      2. Sun XL ,
      3. Yip W ,
      4. Wagstaff A , et al
      . The impact of a pay-for-performance scheme on prescription quality in rural China: impact evaluation. Policy Research Working Paper No. 6892. Washington, DC: World Bank, 2014.
    49. 49.↵
      2. Tang S-nv ,
      3. Jian W-yan ,
      4. Yip WC-M , et al
      . [Impact of primary care oriented outpatient benefit package design in new rural cooperative medical system on hypertension outpatient services utilization]. Beijing Da Xue Xue Bao Yi Xue Ban 2014;46:445–9.
      OpenUrl
    50. 50.↵
      2. Tang Y ,
      3. Zhang X ,
      4. Yang C , et al
      . Application of propensity scores to estimate the association between government subsidy and injection use in primary health care institutions in China. BMC Health Serv Res 2013;13:183.doi:10.1186/1472-6963-13-183
    51. 51.↵
      2. Tanner JH
      . Improving coverage and utilization of maternal and child health services in Lao PDR: impact evaluation of the community nutrition project, in IEG working paper. 2015, independent evaluation group. Washington, DC: World Bank Group, 2015.
    52. 52.↵
      2. Thanh NX ,
      3. Löfgren C ,
      4. Phuc HD , et al
      . An assessment of the implementation of the health care funds for the poor policy in rural Vietnam. Health Policy 2010;98:58–64.doi:10.1016/j.healthpol.2010.05.005
      OpenUrlPubMed
    53. 53.↵
      2. Vellakkal S ,
      3. Gupta A ,
      4. Khan Z , et al
      . Has India's national rural health mission reduced inequities in maternal health services? A pre-post repeated cross-sectional study. Health Policy Plan 2017;32:79–90.doi:10.1093/heapol/czw100
      OpenUrlCrossRefPubMed
    54. 54.↵
      2. Wei X ,
      3. Yang N ,
      4. Gao Y , et al
      . Comparison of three models of ownership of community health centres in China: a qualitative study. J Health Serv Res Policy 2015;20:162–9.doi:10.1177/1355819615579700
      OpenUrlCrossRefPubMed
    55. 55.↵
      2. Wei X ,
      3. Yin J ,
      4. Wong SYS , et al
      . Private ownership of primary care providers associated with patient perceived quality of care: a comparative cross-sectional survey in three big Chinese cities. Medicine 2017;96:e5755.doi:10.1097/MD.0000000000005755
    56. 56.↵
      2. Wong MCS ,
      3. Wang HHX ,
      4. Wong SYS , et al
      . Performance comparison among the major healthcare financing systems in six cities of the pearl River delta region, mainland China. PLoS One 2012;7:e46309.doi:10.1371/journal.pone.0046309
    57. 57.↵
      2. Yip W ,
      3. Hsiao WC
      . Non-evidence-based policy: how effective is China's new cooperative medical scheme in reducing medical impoverishment? Soc Sci Med 2009;68:201–9.doi:10.1016/j.socscimed.2008.09.066
      OpenUrlCrossRefPubMed
    58. 58.↵
      2. Yip W ,
      3. Powell-Jackson T ,
      4. Chen W , et al
      . Capitation combined with pay-for-performance improves antibiotic prescribing practices in rural China. Health Aff 2014;33:502–10.doi:10.1377/hlthaff.2013.0702
      OpenUrlAbstract/FREE Full Text
    59. 59.↵
      2. Asante A ,
      3. Price J ,
      4. Hayen A , et al
      . Equity in health care financing in low- and middle-income countries: a systematic review of evidence from studies using benefit and financing incidence analyses. PLoS One 2016;11:e0152866.doi:10.1371/journal.pone.0152866
    60. 60.↵
      2. Lagarde M ,
      3. Palmer N
      . The impact of user fees on access to health services in low- and middle-income countries. Cochrane Database Syst Rev 2011;(4).doi:10.1002/14651858.CD009094
    61. 61.↵
      2. Li H ,
      3. Qian D ,
      4. Griffiths S , et al
      . What are the similarities and differences in structure and function among the three main models of community health centers in China: a systematic review. BMC Health Serv Res 2015;15:504.doi:10.1186/s12913-015-1162-z
    62. 62.↵
      2. Nachtnebel M ,
      3. O'Mahony A ,
      4. Pillai N , et al
      . Effectively engaging the private sector through vouchers and contracting – a case for analysing health governance and context. Soc Sci Med 2015;145:193–200.doi:10.1016/j.socscimed.2015.05.021
      OpenUrl
    63. 63.↵
      2. Witter S
      . Paying for performance to improve the delivery of health interven-tions in low-and middle-income countries 2012.
    64. 64.↵
      2. Wiysonge CS ,
      3. Paulsen E ,
      4. Lewin S , et al
      . Financial arrangements for health systems in low-income countries: an overview of systematic reviews. Cochrane Database Syst Rev 2017;8.doi:10.1002/14651858.CD011084.pub2
    65. 65.↵
      2. Yip WC-M ,
      3. Hsiao W ,
      4. Meng Q , et al
      . Realignment of incentives for health-care providers in China. Lancet 2010;375:1120–30.doi:10.1016/S0140-6736(10)60063-3
      OpenUrlCrossRefPubMedWeb of Science
    66. 66.↵
      2. Yuan B ,
      3. He L ,
      4. Meng Q , et al
      . Payment methods for outpatient care facilities. Cochrane Database Syst Rev 2017;3.doi:10.1002/14651858.CD011153.pub2
    67. 67.↵
      2. Jan S ,
      3. Laba T-L ,
      4. Essue BM , et al
      . Action to address the household economic burden of non-communicable diseases. The Lancet 2018;391:2047–58.doi:10.1016/S0140-6736(18)30323-4
      OpenUrl
    68. 68.↵
      2. Shrime MG ,
      3. Dare AJ ,
      4. Alkire BC , et al
      . Catastrophic expenditure to pay for surgery worldwide: a modelling study. Lancet Glob Health 2015;3 Suppl 2:S38–S44.doi:10.1016/S2214-109X(15)70085-9
      OpenUrl
    69. 69.↵
      2. Wagstaff A ,
      3. Flores G ,
      4. Hsu J , et al
      . Progress on catastrophic health spending in 133 countries: a retrospective observational study. Lancet Glob Health 2018;6:e169–79.doi:10.1016/S2214-109X(17)30429-1
      OpenUrl
    70. 70.↵
      2. Frenk J
      . The global health system: strengthening National health systems as the next step for global progress. PLoS Med 2010;7:e1000089.doi:10.1371/journal.pmed.1000089
    71. 71.↵
      2. Moon S ,
      3. Szlezák NA ,
      4. Michaud CM , et al
      . The global health system: lessons for a stronger institutional framework. PLoS Med 2010;7:e1000193.doi:10.1371/journal.pmed.1000193
    72. 72.↵
      2. Keusch GT ,
      3. Kilama WL ,
      4. Moon S , et al
      . The global health system: linking knowledge with action – learning from malaria. PLoS Med 2010;7:e1000179.doi:10.1371/journal.pmed.1000179
    73. 73.↵
      2. Mueser PR ,
      3. Troske KR ,
      4. Gorislavsky A
      . Using state administrative data to measure program performance. Rev Econ Stat 2007;89:761–83.doi:10.1162/rest.89.4.761
      OpenUrlCrossRefWeb of Science
    74. 74.↵
      2. Dimick JB ,
      3. Ryan AM
      . Methods for evaluating changes in health care policy. JAMA 2014;312:2401–2.doi:10.1001/jama.2014.16153
      OpenUrlCrossRefPubMed
    75. 75.↵
      2. Hemming K ,
      3. Haines TP ,
      4. Chilton PJ , et al
      . The stepped wedge cluster randomised trial: rationale, design, analysis, and reporting. BMJ 2015;350.doi:10.1136/bmj.h391
    76. 76.↵
      2. Cole ZD ,
      3. Donohoe HM ,
      4. Stellefson ML
      . Internet-based Delphi research: case based discussion. Environ Manage 2013;51:511–23.doi:10.1007/s00267-012-0005-5
      OpenUrl

    Footnotes

    • Handling editor Valery Ridde

    • Contributors This work was conducted as part of an international consortium examining primary health care systems in low-income and middle-income countries led by Ariadne Labs. DP and other members of the consortium developed the broad concept for this work. DP, RD, AP, SJ and BA developed the specific question answered in this paper. RD and AP reviewed the literature for study inclusion and extracted data from included studies. BA and TG wrote the initial version the manuscript. All authors provided input into the analysis and rewriting of the manuscript. All authors approved submission of this work.

    • Funding This publication is based on research funded by Ariadne Labs through Brigham and Women’s Hospital, which is the recipient of a Bill and Melinda Gates Foundation grant. The findings and conclusions contained within are those of the authors and do not necessarily reflect positions or policies of the Bill and Melinda Gates Foundation.

    • Competing interests SA is the editor in chief of BMJ Global Health but was not involved in the evaluation or peer-review process of this article. The other authors declare no competing interests for this work.

    • Patient consent for publication Not required.

    • Ethics approval Ethics approval for the e-Delphi process was granted by the University of New South Wales Human Research Ethics Advisory Panel Executive (Ref HC180135).

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data availability statement All data relevant to the study are included in the article or uploaded as supplementary information.