Article Text
PDF
Abstract
Performance-based financing (PBF) is a mechanism to improve the quality and the utilisation of health benefit packages. There is a dearth of economic evaluations of PBF in the ‘real world’. Afghanistan implemented PBF between 2010 and 2015 and evaluated the programme using a pragmatic cluster-randomised control trial. We conducted a cost-effectiveness analysis of the PBF programme in Afghanistan, compared with the standard of care, from the provider payer’s perspective. The incremental cost-effectiveness ratio of PBF compared with the standard of care was US$1242 per disability-adjusted life year averted; not cost-effective when compared with an opportunity cost threshold of US$349. Incentive payments were the main contributor to PBF financial cost (70%) followed by data verification (23%), staff time (5%) and administration (2%). The unit cost per case of antenatal care (ANC), skilled birth attendance (SBA) and postnatal care (PNC) services in the standard of care was US$0.96 (95% CI 0.92–1.0), US$4.8 (95% CI 4.1–6.3) and US$1.3 (95% CI 1.2–1.4), respectively, whereas the cost of ANC, SBA and PNC services per case in PBF areas were US$4.72 (95% CI 4.68–5.7), US$48.5 (95% CI 48.0–52.5) and US$5.4 (95% CI 5.1–5.9), respectively. To conclude, our study found that PBF, as implemented in the Afghan context, was not the best use of funds to strengthen the delivery of maternal and child health services. The cost-effectiveness of alternative PBF designs needs to be appraised before using PBF at scale to support health benefit packages. PBF needs to be considered in the context of funding the range of constraints that inhibit health service performance improvement.
- health economics
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Key questions
What is already known?
In the last decade, there has been extensive investment in and implementation of performance-based financing (PBF) in low-income and middle-income countries, specifically post-conflict settings, as an approach to expand and improve service delivery.
To date, there have only been two studies reporting on the cost-effectiveness of PBF for health services, with inconclusive findings.
What are the new findings?
This study found that PBF, as implemented in the Afghanistan context, was not the most cost-effective use of scarce funds within the health sector budget constraint.
What do the new findings imply?
Our research supports the growing evidence highlighting the importance of considering efficiency in the design of the PBF incentive structure, the capacity at the facility level to respond to incentives and the relative importance of services in terms of health outcomes.
The success of PBF as a health system strengthening strategy ultimately remains dependent on a sound understanding of context-specific constraints and may require complementary health system investments.
Introduction
Performance-based financing (PBF) has received considerable policy attention in recent years in low-income and middle-income countries (LMICs), as a means to improve health system performance as part of Universal Health Coverage.1 PBF provides incentives to service providers (facilities and workers) when they achieve predefined performance targets.2 A critical concern is whether the overall costs of PBF, including transaction costs of setting up the payment and information systems required, have a greater impact than other direct forms of funding health services (such as budgets). Despite extensive implementation of PBF programmes to improve maternal and child health (MCH) services, to date, there are only two studies of the impact of PBF in LMICs.3 4 Borghi et al3 found that the PBF programme in Tanzania was not cost-effective, whereas Zeng et al4 concluded that PBF was a cost-effective intervention in Zambia.
The Ministry of Public Health (MoPH) of Afghanistan implemented PBF to fund its Basic Package of Health Services (BPHS) between 2010 and 2015 aiming to strengthen the performance of MCH services. The BPHS provides uniform preventive and curative primary health services. The main elements of the BPHS are maternal and newborn care, child health and immunisation, public nutrition, communicable diseases treatment, mental health, disability and physical rehabilitation services and regular supply of essential drugs.5
This study examines the ‘real world’ cost-effectiveness of PBF in Afghanistan ex-post, using data from a pragmatic randomised control trial, using a decision-analytic model, that estimates the cost per disability-adjusted life year (DALYs) averted. To our knowledge, this is the first cost-effectiveness study of PBF in a fragile conflict-affected state.
We aim to contribute to the broad evidence base informing LMICs on whether PBF can extend and improve the performance of health benefits packages in a cost-effective way.
Methods
Study setting
Over the past four decades, Afghanistan has experienced political instability, civil war and pervasive conflict.6 A new democratic government was established in December 2001. In 2003, the BPHS was introduced to provide primary healthcare services, specifically MCH services, to the population. BPHS were contracted to non-state providers (NSPs) in 31 provinces, whereas the MoPH managed the BPHS in the remaining three provinces through direct implementation called Ministry of Public Health Strengthening Mechanism (MoPH-SM).7 PBF was implemented in nine provinces supported by NSPs and two provinces supported by MoPH-SM, covering 463 BPHS health facilities out of 1892 nationally in 2010.8
Intervention description
PBF provided financial incentives to health workers based on the increased production of MCH services (antenatal care (ANC), skilled birth attendance (SBA) and postnatal care (PNC)) above the baseline for each BPHS health facility. These incentives were paid in addition to routine salaries and funding of the health facility. Services were monitored using data from the Health Management Information System (HMIS) and verified through household visits, comparing reported visits with those noted in the health facility registers. If the community validation rate exceeded 80%, then the health facility was entitled to a performance payment. The payments were weighted according to the quality of care assessed by a quarterly score on the national monitoring checklist.9 The indicators and level of performance payments are provided in the online supplemental table S3.
Supplemental material
PBF was evaluated through a large-scale pragmatic cluster randomised trial, details of which are reported elsewhere.5 All facilities within each province were stratified by type of facility and then matched based on the outpatient utilisation rate. Within each matched pair, health facilities were randomly assigned to control and treatment groups. The treatment group received PBF in addition to routine funding, whereas the control group received only their routine salaries.9 The evaluation of PBF involved two household surveys conducted at baseline in 2010 and end-line in 2015 in the catchment area of a sample of treatment and control health facilities. The impact evaluation found that, on average, PBF improved the utilisation of services in all the payment triggering indicators; however, no statistically significant differences were found between study arms. In addition, there was an overall performance score, which was an average of the scores on the 20 indicators from five domains, which were client and community, human resources, physical capacity, service provision and management systems. Each indicator was given equal weighting to develop the composite score that ranged from 0 to 100. Facilities exposed to PBF achieved a statistically higher index score compared with the control group on quality of care.10
Economic evaluation framework
We compared the PBF ‘treatment’ with the standard of care ‘control’ for the population of Afghanistan, assessing cost-effectiveness using incremental cost per DALYs averted. We used a time horizon of 1 year from the start of pregnancy for a hypothetical cohort of women attending BPHS in Afghanistan between 2010 and 2015. We used a decision tree model to estimate DALYs averted by PBF for both mothers and neonates as the trial reported intermediate health outcomes (ANC, SBA and PNC). We parameterised the model with primary cost data, service data from the trial and secondary data. We assessed cost-effectiveness from a provider perspective (MoPH, donor), as data on patient incurred costs were not available.
Model design
We adapted a validated model called the ‘Maternal Health Policy Model’ to evaluate the costs and health outcomes of the PBF intervention in Afghanistan (online supplemental figure S2). This model simulates the natural history, events and service utilisation related to pregnancy and childbirth, including ANC, delivery, abortion, complications and PNC.11 Adapting this model, we developed two decision trees, one for pregnant mothers and one for newborns, to predict incremental costs and health outcomes (online supplemental figure S3). The decision trees were parameterised on the probability of care-seeking and events occurring and associated costs collected during the trial. Within the maternal decision tree, pregnant women have the option to use or not use ANC services, to proceed to delivery with or without a skilled birth attendant or have an abortion, to incur potential complications of pregnancy (ie, haemorrhage, obstructed labour, sepsis, hypertensive disorders, fistula, anaemia and infertility) and to use or not use PNC services. In the neonatal decision tree, newborns may receive PNC and may develop complications (ie, low birth weight, neonatal sepsis and birth asphyxia).
Cost parameter estimation
Estimating the cost of implementing the PBF intervention
We estimated the financial and economic costs of implementing PBF, using primary cost data from Afghanistan. Financial costs included the PBF managers’ salary, incentive payments, data verification and PBF project administration cost. Economic cost added the financial cost with the opportunity costs of items that were used for the purpose of the PBF implementation at the national and provincial level by the MoPH and NSPs (shared staff, building, equipment and transport costs), but were not directly paid for by the PBF scheme. Financial cost data were obtained from PBF project accounts and financial reports. Where PBF costs were shared with other interventions and activities, we allocated costs to PBF using a variety of allocation factors, further details can be found in the online supplemental appendix section on costs. The cost of personnel whose salaries were not solely funded by the PBF project was allocated based on the proportion of their time spent on PBF-related activities. For shared building space, we allocated according to the percentage of floor space used for the PBF intervention. The cost of transportation was estimated based on the consumption of fuel used for the PBF compared with other activities. As the MoPH owned buildings, we used estimates of rent of equivalent building spaces to determine building prices. All (non-building) capital costs were annuitised using a 3% discount rate and life span of 5 years.
Estimating the unit cost of providing services incentivised by PBF
We estimated the financial costs of the services supported by PBF and those in the standard of care. We estimate the unit costs of services. We therefore first estimated unit costs of service delivery in terms of staff time, medical supplies, drugs and capital items. We conducted primary data collection, using microcosting methods (see the online supplemental appendix for more details) in a random sample of 25 health facilities from the 463 BPHS health facilities where the PBF intervention was implemented to estimate the unit costs of services. These costs, minus PBF incentives and support, were used to estimate service costs in the standard of care, as we had no access to standard care facilities. Unit costs were estimated using a micro-costing method including the costs of salaries, drugs, equipment and building. Interviews were undertaken with all staff (206) at sampled health facilities to determine the proportion of staff time spent on each service and the proportional use of equipment for each service. The percentage of floor space used for each service was measured. The average costs of drugs and supplies used were calculated using the list of prescribed medicine for each service and the pharmacy register book at each of the health facilities.
Estimating incremental costs of the PBF
The total cost of services was estimated in the decision-analytic model by adding the unit cost for each of the services (for PBF including PBF costs) received by the cohort in the decision-analytic model. Prices in local currency were converted to US dollars to allow comparisons between countries (US$1 to 58 Afghani in 2015). We used secondary data from the literature to estimate unit costs of maternal and newborn-related complications.11 All costs were inflated and adjusted to 2015 US dollars.
Service utilisation and outcomes
Given the PBF impact evaluation shows an increase in the utilisation of services and significant improvement in the quality of care in the PBF intervention (treatment) compared with the standard of care (control); we used utilisation rates of ANC, SBA and PNC services derived from the PBF impact evaluation10 to estimate the relative effect of PBF intervention on the utilisation of these services. We also sourced other data from the literature.11–15All parameters used in the decision-analytic model are presented in table 1.
Model parameters
We estimated the incremental cost-effectiveness ratio (ICER) defined as the incremental cost of PBF per DALY averted compared with the standard of care. The ICER was assessed against the US$349 per capita threshold estimated by Wood et al’s for Afghanistan,16 that is higher than 0.5 of gross domestic product per capita in 2015 (US$570), as no more recent estimates were available. Given that 1 DALY averted equals 1 year of healthy life lived, we translated the total number of DALYs averted by the intervention to an annual reduction in maternal and neonatal deaths. We used a 3% discount rate on cost and DALYs averted in our primary analysis. We undertook a series of one-way sensitivity analysis across key model parameters varying each parameter at a time up to ±30% of the base case value. In addition, we used probabilistic sensitivity analysis (Monte Carlo situation of 1000 times) to assess parameter uncertainty and produce a 95% CI around the ICER.
Patient and public involvement
Patient and the public were not involved in the design and other stages of this study. However, we plan to disseminate the findings through publication and presentations in conferences and public events.
Results
The total incremental financial and economic provider costs of the PBF programme were US$10 677 465 and US$11 896 380, respectively, during the 6-year life of the scheme, as shown in table 2. Incentive payments were the main contributor to economic costs (63%) followed by HMIS data verification (21%), administration (10%) and staff time (6%). The online supplemental table S4 shows the PBF annual financial programme cost and the online supplemental table S5 presents the annual breakdown of the administration costs.
The cost of implementing the PBF programme over a 6-year period (US$ 2015)
Table 3 presents the average annual health facility cost of providing ANC, SBA and PNC services in the PBF and control areas. The average annual costs for PBF facilities of providing ANC, SBA and PNC services were US$20 607 (95% CI 14 359–26,818) compared with US$2168 (95% CI 2073–2438) in control facilities. Table 4 shows the unit cost of services in the PBF and control groups. The estimated unit costs per case receiving ANC, SBA and PNC services in the control group were US$0.96 (95% CI 0.92–1.0), US$4.76 (95% CI 4.1–6.3) and US$1.3 (95% CI 1.2–1.4), respectively, whereas the costs of ANC, SBA and PNC services per case in the treatment group were US$4.72 (95% CI 3.6–5.8), US$48.5 (95% CI 48.0–52.5) and US$5.4 (95% CI 5.1–5.9), respectively. The costs of incentives and data verification were the main driver for the higher unit costs in the PBF facilities. The annual per capita cost of the PBF programme was US$0.4 (95% CI 0.2–0.6) in the study areas as presented in the online supplemental table S6.
Average cost of PBF indicators in control and treatment groups per health facility per year (US$ 2015)
The unit cost of delivering selected maternal and neonatal health services for facilities in control and treatment groups (US$ 2015)
The incremental cost of the PBF programme per DALY averted was US$1241 (table 5). The probabilistic sensitivity analysis at US$1228 (95% CI 1118–1337) found that 0% of simulations the ICER lay below the opportunity cost threshold (not cost-effective). In total, 13 028 incremental DALYs (5658 incremental maternal DALYs and 7370 incremental neonatal DALYs) were averted that corresponds to an incremental 253 deaths averted (138 maternal and 115 neonatal) between 2010 and 2015 (equivalent to an annual reduction in maternal mortality of 29 deaths per 100 000 live births and neonatal mortality of 0.24 deaths per 1000 live births), across the 11 provinces of Afghanistan, with a total population of 4.06 million living in the coverage area of PBF facilities. Figure 1 presents the PBF cost-effectiveness acceptability curve, figure 2 the cost-effectiveness plane and online supplemental tables S7 and S8 show the results of one-way and two-way sensitivity analyses.
Annual costs and outcomes for mothers and newborns in PBF and control areas and the incremental cost-effectiveness of the PBF programme (US$ 2015)
PBF cost-effectiveness acceptability curve. GDP, gross domestic product; PBF, performance-basedfinancing; WTP, willingness to pay.
Cost-effectiveness plane of the PBF treatment group compared with the control group. DALY, disability-adjusted life years; ICER, incremental cost-effectiveness ratio; PBF, performance-based financing.
Discussion
The focus on maternal and neonatal health during pregnancy, childbirth and postpartum period is proved to be effective in reducing maternal and neonatal morbidity and mortality.17 18 Afghanistan implemented a PBF intervention on a large scale aiming to improve MCH services. The PBF intervention was evaluated through a pragmatic cluster randomised trial. We developed a decision-analytic model of the care pathways, cost and outcomes for pregnant women and newborns to estimate the incremental cost-effectiveness of the PBF scheme relative to the standard of care. Although the PBF intervention resulted in an improvement in the quality of MCH services in the PBF group, our study found that this initiative was unlikely to be cost-effective from a provider payer’s perspective. The finding supports the study from Tanzania that the PBF intervention was not cost-effective (which did not measure DALYs averted), despite improvements in utilisation rates,3 but our study finding contradicts the only previous robust study from Zambia in which the PBF intervention was found to be cost-effective.4
Our study found that the costs of implementing PBF were substantial compared with those in the non-PBF control districts. PBF cost was primarily driven by the payment of incentives and data verification linked to incentive payment (together accounting for 93% of the total financial cost and 84% of the total economic cost). The proportion of incentives contrasts with Zambia and Tanzania. In Zambia, incentives and verification accounted for around 50% of the total cost.4 Borghi et al19 reported a high administration cost (financial 63% and economic 78%) in Tanzania when implementing a PBF intervention.
There are a number of possible explanations for the lack of cost-effectiveness of PBF to support the basic package for MCH services in Afghanistan. First, the theory of change that supported the design of PBF in Afghanistan posited that high levels of financial incentives would motivate healthcare workers to improve the quality of care and subsequently increase demand for MCH services.5 Although incentives may influence providers, availability of resources such as sufficient healthcare workers, equipment, essential drugs and supplies, and effective referral systems are essential to ensure the quality provision of services.20 However, neither the PBF programme was able to overcome these systemic resource constraints, nor the health facilities had financial autonomy to procure them locally.21 In Zambia, health workers had significant autonomy in addressing the shortage of essential inputs in health facilities.4
Second, the level of incentive is critical to PBF efficiency.22 When too low, an incentive might fail to result in behaviour change, whereas when too high, an incentive consumes resources unnecessarily. In Afghanistan’s PBF scheme, the incentive payment for delivery with SBA was set at US$37 per case. In contrast, the incentives for ANC and PNC were set much lower, at US$2.8 per case. The SBA incentive made 72% of the overall cost of the PBF incentives, and it consumed 51% of the PBF total financial cost, whereas the ANC and PNC incentives constituted only 17% and 9% of the PBF total financial cost. Although the substantial effect of SBA on proper management of delivery and prevention of pregnancy complications is critical, both ANC and PNC are also important. ANC has a positive effect on the identification of pre-eclampsia/eclampsia and prevention of anaemia,23 and PNC plays a crucial role in early identification and appropriate referral of maternal and newborn complications, family planning and promotion of healthy behaviours for mother and newborn. Both are designed to encourage a sustained relationship between health services and the mother during pregnancy. It is also important to note that 66% of global maternal mortality happens in the postpartum period, and the first 24 hours after delivery is crucial given 45% of deaths occur in this time.17 Therefore, although the incentives set may have reflected a level of workload, it is essential to also consider the value of services in terms of their contribution to health outcomes, and not comparatively disincentivise those services with low or no additional payments.
Third, even though HMIS data verification is a crucial element of PBF interventions, this ‘transaction cost’ can be relatively high. In our case, the verification was 23% of the total financial cost and 21% of the overall economic cost of the PBF intervention in Afghanistan. This cost could be reduced by testing different modalities. For example, Zambia replaced the monthly verification of all health facilities with a risk-based model in which health facilities were assessed based on the expected risk of misrepresenting data.4
Our study has some limitations. The main limitation is that we had to estimate health service costs in the non-PBF sites, using micro-costing from the PBF sites. This was done based on the assumption that underlying service costs and expenditures from other sources would be balanced between intervention and control sites due to the randomised design and similar funding to all sites from other funders (including government payment of staff salaries). Although there may have been some fungibility at the facility level away from MCH services in PBF sites, biasing our PDF site upwards, this will have been minimal and only apply to non-salary items. Further, the PBF impact evaluation shows that NSPs, in implementing the PBF programme, demonstrated different capacities that might have affected both cost and effectiveness. However, aside from the control design, we did not have additional data to explore the impact of these differences on cost-effectiveness. On the effect side, there may have been a spillover effect from the PBF group into the control group due to the location of both facilities in the same province and the movement of staff and the population across facilities. Control health facilities were likely aware of PBF and tried to compete with treatment health facilities on performance. An additional limitation is that the estimation of staff time spent on each service and proportional use of equipment for each service was based on self-reported interviews with health facility staff that may be prone to bias. We also had to source some data regarding maternal and neonatal-related complications from outside the study.11–15 These parameter limitations were addressed by applying a wide-range sensitivity analysis.
To conclude, our study found that PBF, as implemented in the Afghan context, was not the best use of funds to strengthen the delivery of MCH services. It is likely that the amounts provided were too low for some services, there was insufficient flexibility in using those resources to address service performance constraints and data verification was not economically efficient. Further research into the efficiency and cost-effectiveness of PBF schemes with different designs in different settings is important to ensure that PBF improves performance and inform how best to strategically purchase health benefit packages in LMICs in order to make progress towards UHC.
Footnotes
Handling editor Lei Si
Twitter @BlanchetKarl
Contributors All authors participated in the production of the manuscript. ASS conceived and designed the study, conducted data collection and data analysis and wrote the first draft. AV supported the model development and analysis. JB, KB and AV provided analytic feedback and contributed to writing the manuscript. All authors read and approved the final manuscript.
Funding This study was conducted as part of a self-funded PhD by the first author.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement All data relevant to the study are included in the article or uploaded as supplementary information. All data relevant to the study are included in the article or uploaded as supplementary information.