You are here

Article Text

Article menu

Download PDFPDF

XML
Research
Social return on investment of emergency obstetric care training in Kenya
  1. Aduragbemi Banke-Thomas1,2,
  2. Barbara Madaj1,
  3. Nynke van den Broek1
  1. 1 Centre for Maternal and Newborn Health, Liverpool School of Tropical Medicine, Liverpool, UK
  2. 2 Department of Health Policy, London School of Economics and Political Science, London, UK
  1. Correspondence to Dr Aduragbemi Banke-Thomas; a.banke-thomas{at}lse.ac.uk

Abstract

Introduction Emergency obstetric care (EmOC) training is considered a key strategy for reducing maternal and perinatal morbidity and mortality. Although generally considered effective, there is minimal evidence on the broader social impact and/or value-for-money (VfM). This study assessed the social impact and VfM of EmOC training in Kenya using social return on investment (SROI) methodology.

Methods Mixed-methods approach was used, including interviews (n=21), focus group discussions (n=18) incorporating a value game, secondary data analysis and literature review, to obtain all relevant data for the SROI analysis. Findings were incorporated into the impact map and used to estimate the SROI ratio. Sensitivity analyses were done to test assumptions.

Results Trained healthcare providers, women and their babies who received care from those providers were identified as primary beneficiaries. EmOC training led to improved knowledge and skills and improved attitudes towards patients. However, increased workload was reported as a negative outcome by some healthcare providers. Women who received care expected and experienced positive outcomes including reduced maternal and newborn morbidity and mortality. After accounting for external influences, the total social impact for 93 5-day EmOC training workshops over a 1-year period was valued at I$9.5 million, with women benefitting the most from the intervention (73%). Total direct implementation cost was I$745 000 for 2965 healthcare providers trained. The cost per trained healthcare provider per day was I$50.23 and SROI ratio was 12.74:1. Based on multiple one-way sensitivity analyses, EmOC training guaranteed VfM in all scenarios except when trainers were paid consultancy fees and the least amount of training outcomes occurred.

Conclusion EmOC training workshops are a worthwhile investment. The implementation approach influences how much VfM is achieved. The use of volunteer facilitators, particularly those based locally, to deliver EmOC training is a critical driver in increasing social impact and achieving VfM for investments made.

  • emergency obstetric care
  • training
  • social return on investment
  • social impact
  • value for money

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/bmjgh-2018-001167

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Key questions

    What is already known?

    • Training in emergency obstetric care (EmOC) leads to increased knowledge and skills among healthcare providers and improved pregnancy outcomes.

    • There has been a strong case for continued investment in EmOC trainings; however, there has been limited evidence on its cost-effectiveness and more broadly value-for-money (VfM).

    What are the new findings?

    • Taking a broader societal perspective, the stakeholders who benefit the most from investments in training healthcare providers in EmOC are women in need of such care.

    • EmOC training guarantees VfM in all scenarios except when trainers are paid consultancy fees and the least amount of outcomes of the intervention occurs.

    What do the new findings imply?

    • EmOC training is a worthwhile investment. To increase the VfM derived from it, options such as converting daily subsistence allowance (DSA)/per diem budgets to financial aid for improving salaries and working conditions of healthcare providers, harmonisation of DSA/per diem based on output following training and payment of DSA/per diems based on actual need rather than hierarchy are some of the alternatives that can be explored.

    • Use of volunteer trainers, particularly those who work locally, to deliver EmOC training is a critical driver in increasing social impact and achieving VfM for investments made.

    Introduction

    Globally, 280 000 women still die annually due to complications of pregnancy and childbirth, with more than 99% of these deaths occurring in low-income and middle-income countries (LMICs).1 In addition, 2.6 million babies are delivered stillborn and another 2.7 million die during the first 28 days of life.2 3 In Kenya, as in other LMICs, complications during or after pregnancy or at the time of birth are major causes of morbidity and mortality, resulting in an estimated 362 maternal deaths per 100 000 live-births and 23 newborn deaths per 1000 live-births.4 Emergency obstetric care (EmOC) provided by a skilled birth attendant in a timely fashion can significantly reduce pregnancy-related morbidity and mortality.5 Previous estimates suggest that when EmOC is performed by a competent healthcare provider, intrapartum stillbirths may be reduced by between 45% and 75%,6 and institutional maternal mortality may be reduced by between 15% and 50%.7

    Training in EmOC has been shown to lead to increased knowledge and skills among healthcare providers and improved pregnancy outcomes for women who receive care from them.8–10 Thus, while a strong case can be made for continued investments based on evidenced effectiveness,9–11 there is a need for, and a growing interest in, evidence to demonstrate if EmOC training is a cost-effective intervention and, more broadly, if investments made in implementing such training can be considered value-for-money (VfM).12 13 Traditionally, cost-effectiveness, cost-utility and cost-benefit analyses have been used to assess VfM of public health interventions.13 However, more recently, social return on investment (SROI), which is an expanded form of the conventional cost-benefit analysis, with capacity to capture both positive and negative impacts of an intervention and including the beneficiaries valuation of impact, has been promoted as a more comprehensive tool for assessing VfM.13 SROI captures the broader social value of interventions including from multiple stakeholders’ perspectives and relates this to the cost of the intervention.14 As such, SROI will be relevant in demonstrating the impact of EmOC training, an intervention which is typically expected to yield benefits beyond those for trained healthcare providers.10 However, to date no such study has been conducted.15 This study sought to assess the social impact and VfM of EmOC training implemented in Kenya, using the SROI methodology.

    Methods

    An abridged 30-item version of the SROI checklist included in the most up-to-date guidelines for conducting SROI studies was used to guide reporting of this study. The items included in this abridged version were those focused on reporting of the study and not conduct dissemination or use of findings (online supplementary table 2).14

    Supplemental material

    The intervention—EmOC training

    Five-day training workshops were delivered by a mix of international (UK-based) and national (Kenyan) facilitators, who had experience working in low-resource settings. Training content was adapted to specific predefined country needs. The principles of adult education were applied, using a combination of lectures and practical hands-on ‘skills and drills’ sessions to enhance effective learning while encouraging group discussions and mentoring.8 EmOC workshops were conducted across all 47 counties in Kenya, in venues outside the work place of the health workers.

    A theory of change (ToC), which set out how the desired change was expected to happen following implementation,16 was developed as part of the proposal and during EmOC training in Kenya.8 Resources used in implementing the training (including trainers, training venue, manuals, curriculum, equipment and payment for subsistence for trainers and trainees) were expected to result in competent healthcare providers which was expected to lead to improved availability and quality of EmOC for women who needed such care. By ensuring that competent healthcare providers were in place to provide EmOC, this was also expected to lead to an increased demand and uptake of EmOC. These outcomes were then expected to contribute to an overall reduction in maternal and perinatal morbidity and mortality (online supplementary figure 1).8 17

    Supplemental material

    Study design

    SROI can be used to either retrospectively evaluate social impact of outcomes that have already occurred due to the intervention (evaluative-type SROI) or model social impact of anticipated outcomes (forecast-type SROI).14 This study reports an evaluative-type SROI that assessed the social impact and VfM of EmOC training, based on outcomes that occurred over a 1-year period. Underpinned by the SROI principles (online supplementary table 2), the study was conducted aligning with the SROI stages 1–5,14 while leveraging lessons learnt from previous SROI studies of public health interventions.13

    Supplemental material

    Data collection

    A mixed methods approach was used to collect all the data required for the SROI analysis. Key informant interviews (KIIs), paired interviews (PIs) and focus group discussions (FGDs) integrated with a value game were used to collect relevant qualitative data. Analysis of programmatic databases and review of existing literature was used to collect the required quantitative data.

    For SROI stage 1 (Establishing scope and identifying key stakeholders), we conducted a review of the literature,18 and a brainstorming session with programme staff to preidentify stakeholders. These were subsequently purposively sampled, asked to confirm their role and used to identify additional stakeholders (snowballing). FGDs, PIs and KIIs were used to collect data on the role of stakeholders (figure 1). In all, we conducted 28 FGDs, 3 PIs and 18 KIIs with 69 trained healthcare providers, 114 women who received EmOC and their relatives, 30 facilitators and training organisers and 6 healthcare facility managers and Ministry of Health staff.

    Figure 1
    Figure 1

    Data collection methods used for the SROI stages. EmOC, emergency obstetric care; FGD, focus group discussions; KII, key informant interviews; PI, paired interviews; SROI, social return on investment.

    For SROI stage 2, data on inputs, outputs and outcomes were collected. Inputs (direct cost of implementing EmOC training) and outputs of the training were collected from programmatic data linked to the predefined ToC (online supplementary figure 1 ). In addition, stakeholders deemed to be beneficiaries were asked to identify, describe and map outcomes that they had experienced during FGDs or PIs, as both allowed interaction between participants, as opposed to KIIs (figure 1). This helped to identify any modifications to the predefined ToC based on real beneficiary experience.

    For SROI stage 3, stakeholders were asked to establish how long outcomes had lasted for and to place a value on these (figure 1). An adaptation of the value game, based on the contingent valuation technique19 and designed in the form of a visual questionnaire, was used during FGDs and PIs conducted with stakeholders who were deemed as beneficiaries to aid the valuation process in this study. In adapting the value game, beneficiaries were asked to use calibrated benchmarks of items with known market values arranged in ascending order to value the outcomes of the intervention.

    For SROI stage 4, data were collected from beneficiaries on estimates for, ‘what would have happened without the training’ (deadweight), ‘how much of an outcome had displaced other outcomes’ (displacement), ‘how much of the outcome was caused by the contribution of other organisations or people’ (attribution) and ‘deterioration of an outcome over time’ (drop-off).14 Quantitative data were collected at baseline and 1 year follow-up from the programmatic database to demonstrate the magnitude of the change. Changes for outcomes not routinely collected were modelled based on evidence in the literature (figure 1).

    For all KIIs, PIs or FGDs, piloted topic guides were prepared in English and Swahili, ensuring cross-cultural equivalence in translation with the aid of native speakers.20 All discussions and interviews were audio recorded with AudioNote Recorder application (Luminant Software, New York, USA) with parallel descriptive and reflective note-taking done to capture subliminal behaviour of respondents. On the other hand, quantitative data were collected and stored in a Microsoft Excel (Microsoft, Redmond, USA) spreadsheet.

    Data analysis

    Data collected in Stages 1–4 were analysed before incorporation into the SROI model. For SROI stage 1, the Bryson’s basic stakeholder analysis technique21 was used to systematically identify and classify stakeholders as primary beneficiaries (or not) based on their perspectives of their roles and levels of importance and influence.

    For qualitative data, following verbatim transcription of the audio recordings, the thematic approach was used to reduce the data through summarisation and synthesis.22 For this approach, we followed the Braun and Clarke’s six-step approach for qualitative analysis.23 This was done digitally with the aid of NVivo 10 (QSR International, Memphis, USA). Relevant to the SROI analysis, data were extracted specifically with regard to the stakeholder’s perspectives on outcomes and how much of these were specifically due to the intervention. Median percentage attribution and valuation of the various stakeholder-described outcomes obtained from the value game were calculated. These values were then triangulated with evidence from the existing literature.

    For input, a bottom-up costing approach was used to calculate the direct cost of implementing EmOC training in Kenya for a year. This approach was chosen because it is known to be more accurate compared with the alternative top-bottom costing approach.24 Direct costs of implementation included for the analysis were costs of training venue, training manual, training equipment, daily subsistence allowance (DSA) for trainees, DSA for trainers (National and UK-based faculty) and travel support for UK-based faculty.15

    For output, the number of trained healthcare providers across Kenya and financial valuation of each outcome for the 1-year period of study based on insight from the value game and existing evidence was computed. Financial valuation of each outcome was then reduced to reflect the effect of external influences (median percentage attribution, drop off and deadweight). Following the deductions, financial valuations of all outcomes were summed up (SROI Stage 5). This total outcome valuation was then divided by the total value of the input (direct cost of training implementation). The SROI ratio and the net SROI ratio were calculated as:

    Embedded Image

    Embedded Image

    Depending on data sources, initial financial analyses were done in the Kenyan national currency—Kenyan Shillings (KSh) and Great Britain Pounds (£), triangulating values from the Central Bank of Kenya and OANDA for the year of implementation, 2014—(1.00KSh=£0.0068; £1.00=145.50 KSh).25 26 All financial values were subsequently converted to and presented in International Dollars (I$), using Purchasing Power Party factors, to aid comparison with other assessments.15

    Multiple one-way sensitivity analyses for each variable (input or output) and two-way sensitivity analyses that combined input and outcome scenarios were conducted to test assumptions used in the model.

    All quantitative and financial data analyses were done in Microsoft Excel (Microsoft, Redmond, USA).

    Ethics statement

    Written informed consent was obtained from respondents who agreed to partake in the study. No financial incentive was given to participants. All responses were anonymised.

    Results

    The results of the analyses are presented by SROI stage.

    Stage 1 results: Stakeholder analysis

    The Department for International Development (DFID/UKAid) provided funding to implement the EmOC training in Kenya. The Centre for Maternal and Newborn Health (CMNH) worked with the Government of Kenya (GoK) to develop and adapt the training package and approach. The GoK provided the enabling environment for the training to be implemented. CMNH oversaw the implementation of the training, preparation and allocation of both international and national training facilitators. Trained healthcare providers returned to their healthcare facilities, passed on information to yet to be trained healthcare providers and provided care to women and their babies, who live within families and communities and work and contribute to the wider society.

    Based on discussions with all stakeholders, trained healthcare providers, women and their newborns who received care were classified as primary beneficiaries of EmOC training (figure 2). Justification for inclusion or exclusion of the of stakeholder groups in the SROI impact map is detailed in online supplementary table 3.

    Supplemental material

    Figure 2
    Figure 2

    Stakeholder analysis matrix of EmOC training in Kenya classification based on interviews and discussions with key stakeholders. (A) The primary beneficiaries; (B) the ones that can make the difference; (C) the risk group; (D) the bystanders. CMNH, Centre for Maternal and Newborn Health; DFID, Department for International Development; EmOC, emergency obstetric care.

    Stage 2: mapping outcomes of EmOC training

    Trained healthcare providers reported that training led to improved knowledge for self and colleagues and improved skills which were relevant to practice. Healthcare providers deemed improved attitude to clients as a positive outcome of EmOC training. However, some healthcare providers reported increased workload (because more women accessed care) as a negative outcome.

    Healthcare providers observed that the training had ultimately resulted in positive outcomes for pregnant women and babies, including reduced morbidity and mortality associated with pregnancy and childbirth. This was corroborated by women who had received care from healthcare providers. For women, the outcomes expected from care were centred on positive health outcomes; they expected to remain healthy and alive during and after pregnancy and childbirth (No maternal morbidity/mortality), their baby remained healthy and alive during and after pregnancy and childbirth (no neonatal morbidity/mortality/stillbirth) and that any long-term complications of birth such as obstetric fistula were avoided.

    Stage 3: evidencing outcomes and giving them a value

    Indicators and magnitude of change (total number of stakeholders who experienced the outcome) for each outcome are reported in table 1. For obstetric fistula incidence, which was not collected as part of the programme outcomes, 0.08% was used to estimate number of cases that would be avoided with the intervention.27

    View this table:
    Table 1

    Quantity of change for material outcomes by stakeholder group

    When asked about financial valuation of the outcomes experienced, the most prevalent beneficiary response was that the outcomes were ‘invaluable yet inestimable’. However, with the aid of the value game, financial proxies were derived and benchmarked with stated valuation of the items. Summing up all values, the total social impact across all stakeholder groups was I$15 862 821.77 (table 1).

    Stage 4: establishing impact

    Table 1 shows financial proxies used. The median and modal percentage change associated with the intervention as estimated by healthcare providers amounted to 75%. As such, for the base-case scenario, a 25% deadweight was applied. For outcomes related to women and their babies, 0% was used as deadweight, as there was no indication that any increase in receiving EmOC would have occurred without the intervention since all the outcome data used in this study were based on programmatic data collected from healthcare facilities where healthcare providers had received training. The one exception to this was maternal mortality reduction, which beneficiaries all agreed had occurred due to many factors. For this outcome, a 50% attribution percentage factor was used for the base-case scenario (table 1).28 There was no evidence of displacement or drop-off of outcomes reported by beneficiaries of the intervention.

    Stage 5: calculating SROI ratio

    The total direct implementation cost was I$744 774.32. DSA paid to trainees constituted the largest proportion (32%), followed by the cost of training equipment (28%). Printing of training manuals made up approximately 1% of the cost and was the lowest (table 2).

    View this table:
    Table 2

    Direct implementation costs by item

    During the period under review, 93 courses were conducted across 44 training venues in Kenya. From programme data, 2965 healthcare providers were trained. The cost per trained healthcare provider was I$251.16. The cost per trained healthcare provider per day was I$50.23.

    After excluding external influences (attribution and deadweight), the total social impact was valued at I$9 486 477.27. Reduced maternal mortality contributed to the largest proportion of the social impact followed by improved skills of healthcare providers (I$5 559 254.03). Improved attitudes to patients (I$3978.72) and increased workload (I$1029.80) contributed the least to the overall financial valuation of the training outcomes. Women benefited the most from the EmOC training of healthcare providers (I$6 960 519.83 or 73% of total social value), followed by trained healthcare providers (I$2 443 033.02, 26%) and babies (I$81 997.06, 1%) (table 1).

    When the total outcome valuation is divided by the total direct financial implementation costs, the SROI ratio obtained was 12.74:1.00. This means that for every I$1 invested, there was I$12.74 of social value created. The net SROI was estimated at I$11.74 with a payback period of 1 month and 3 days.

    Sensitivity analysis

    Using CI values around estimates of the different variables in the model (financial proxies, attribution and displacement factors), we found no change in our conclusion that the training guaranteed VfM. This provided a degree of confidence over the estimates used for the SROI analysis.

    For training outcomes, the most significant variation in the SROI ratio was around the attribution and financial proxy used for the ‘reduced maternal mortality’ outcome (9.01–16.47). Testing various implementation models, the ratio ranged from 6.46, if all facilitators used were UK-based (international volunteers), to 13.94 if all facilitators were Kenyan (national volunteers) (figure 3).

    Figure 3
    Figure 3

    Tornado diagram showing results of the sensitivity analysis.

    When all the worst-case scenarios relating to the outcomes were combined, the value of outcomes comes to I$1 875 744.01, and the estimated SROI ratio was 2.52, and when the better-cases were all combined, the value of outcomes comes to I$18 460 024.90 and the estimated SROI was 24.79 (figure 3).

    When the best-case scenario for outcomes ($18 460 024.90) is combined with volunteer facilitators who are all locally based in Kenya (I$680 695.16), the estimated SROI comes to 27.12. When the worst-case scenario for outcomes (I$1 875 744.01) is combined with volunteer facilitators who are all based in the UK (I$1 467 372.46), the estimated SROI comes to 1.28 (figure 3). If facilitators were all UK-based and received a standard consultancy fee of US$500/day for the 5-day training, the total training cost comes to (I$2 412 844.96). A scenario combining paid UK-based consultants with the worst-case and best-case scenario for outcomes yielded an estimated of 0.78 and 11.35, respectively (figure 3).

    Discussion

    Main findings

    This study showed that EmOC training resulted in social impact beyond the expected direct or immediate recipients of training (healthcare providers) and that women who received care from trained healthcare providers benefited the most as morbidity was avoided, and lives were saved. In addition, over a 1 year implementation period, EmOC training in Kenya was shown to be a worthwhile investment as it guaranteed VfM, with payback on investment in 1 month. Our analysis showed that for every I$1 invested, there was I$12.74 of social value created.

    Interpretation

    In this study, estimating the cost of implementation and outputs of the intervention were straightforward and easily retrieved from programme accounting data.12 We estimated the cost/trained healthcare provider/day for the 5 day training as I$50.23 per trained healthcare provider per day. This is within the cost range of trainings in LMICs that require accommodation of trainers and/or participants in the literature (I$33 and I$90).15

    Previous studies have shown that EmOC training is a cost-effective way of increasing knowledge and skills of healthcare providers,29 30 their productivity in terms of managing obstetric complications29–32 and for improving maternal outcomes, both in terms of number of lives saved and number of disability-adjusted life years averted.33 In our study, the total financial valuation for 1 year was estimated to be I$9 486 477.27 with women benefiting the most from the intervention (73% of social impact created). Trained healthcare providers made up 26% of the social impact created, while newborns made up just 1% of the social impact. Other literature suggests that, willingness to pay for a child’s life by parents in order to avoid fatality due to asthma, peaks when the child is 4 years old.34 Though it is plausible that the impact of saving the life of a newborn (who then goes on to become an infant and then a child) may be higher in the future, it would be overclaiming to attribute more financial value to EmOC training outcomes related to newborns in this study and this would not be in line with the guiding principles of SROI.14 A forecast SROI that takes a prospective lens would be more useful and would better reflect the long-term ‘benefits’ and ‘value’ of saving a newborn’s life.14 No financial valuation was attributed to stillbirths that were prevented, since the cost of care and hospital stay for stillbirths was already included in those attributed to the care received by the mother. This is, however, counterintuitive and further study is needed to provide data to better model this in future.

    With a return of I$12.74 for every $1 invested, our study demonstrates that EmOC training can be considered to be VfM. As we found no similar analyses for EmOC training reported in the literature, we were unable to compare this finding with others. For comparison with ratios obtained for other maternal health interventions, it is important to use I$ values and compare methodologically similar studies (eg, evaluative-type SROI) and study settings. The only other published evaluative-type SROI study of a maternal health intervention (support programme for pregnant women to improve to healthcare and address issues such as addiction, family violence and poverty) reported a ratio I$8.24:1. No sensitivity analysis was reported by the authors.35 A forecast SROI for a baby-friendly community initiative conducted in Kenya reported a much higher ratio (71:1), but the study included findings that were expected to occur in the future rather than those documented to have occurred.36

    In our study, the various assumptions included in the model were tested using sensitivity analysis, thereby increasing the transparency and reliability of the analysis.37 Based on the actual training implementation approach which used volunteer facilitators, our analyses showed that all scenarios proved to guarantee VfM, even when the least amount of outcomes (worst-case scenario) occurred. However, if the implementation model was altered to use only facilitators recruited from the UK who required paid consultancy fees and, the least amount of outcomes occurred, then the training did not assure VfM, with an SROI ratio of I$0.78 for every I$1 spent on implementing the training. Even if the best-case scenario for outcomes occurred, our findings showed that paid trainers still guaranteed less SROI (11.35) than the base-case scenario (12.74). This has clear implications for policy. The use of teams of experienced specialists from the UK and in-country who serve as volunteer trainers8 makes EmOC training cost-beneficial even in scenarios when the outcomes of the training are not fully realised. As such, governments, non-governmental organisations and other relevant stakeholders should explore innovative ways of engaging specialists to volunteer their time and, in settings providing ongoing training, volunteer time could be included as part of the job description. However, there is an opportunity cost associated with the use of volunteers, as they generally forego income they could otherwise be earning, and the health service may need to find replacements for the time spent away.

    In terms of costing, DSA paid to trainers (26.3%) and trainees (32.1%) made up the largest proportion of funding required. Similar findings have been reported in the literature.15 DSA or ‘per diems’ are repeatedly paid as part of implementation of several MNH initiatives.15 33 38–40 A policy of ‘no per diem payments’ was reported in Zimbabwe and this significantly reduced the cost of training.41 The ethics of per diem payments has been challenged before, with recognition that this leads to a culture where training becomes ‘an opportunity to supplement income’, rather than an opportunity to build professional capacity.41 42 This has also been referred to as ‘acute perdiemitis’.43 Options such as converting DSA/per diem budgets to financial aid for improving salaries and working conditions of healthcare providers, harmonisation of DSA/per diem based on output following training and payment of DSA/per diems based on actual need rather than hierarchy are some of the alternatives that require further exploration. Costs saved from these options can provide the additional funds required to sustain training beyond the life-cycle of the intervention.

    Study strengths and limitations

    To the best our knowledge, this is the first SROI of EmOC training conducted.15 The study used an existing comprehensive database which captured progress and impact of implementation during the entire programme life-cycle. Robust datasets are needed for SROI assessments and these were available and enabled this study to be conducted.14 The use of the contingent valuation technique as part of the value game allowed beneficiaries to value outcomes that would otherwise not have been included.

    However, the sensitivity of the value game is also a limitation.44 Data for financial valuation were mostly obtained from the grey literature. In this study, discussion with in-country experts who had the local knowledge required to decipher the sensibility of the financial proxies retrieved was triangulated with financial valuations obtained from the literature, before incorporating these into the SROI analysis. In addition, specifically for the financial proxies used for valuation of maternal morbidity and mortality, it is reasonable to expect that different obstetric emergencies would potentially have varied social impacts, whereas, for these analyses, it was assumed all types of maternal morbidities had the same social impact.

    Conclusion

    EmOC training can be considered as VfM and has a beneficial SROI ratio. Exploring strategies that help reduce cost and maximise outcomes and making use of volunteer facilitators has been shown to be feasible in practice and will further improve the SROI ratio, thereby guaranteeing even higher returns on the investment made.

    Acknowledgments

    The authors thank Dr Pamela Godia and Ms Judith Maua for their unwavering support in planning this study as well as Mr Josiah Osiri for his support in moderating the focus group discussions in Swahili.

    References

    1. 1.↵
      1. WHO, UNICEF, UNFPA, World Bank Group, The United Nations Population Division
      . Trends in maternal mortality: 1990 to 2015. Geneva: World Health Organization, 2015.
    2. 2.↵
      1. WHO
      . Neonatal mortality: situation and trends. Global health Observatory data. Geneva: World Health Organization, 2016.
    3. 3.↵
      1. WHO
      . Stillbirths. Geneva: World Health Organization, 2016.
    4. 4.↵
      1. KNBS, Ministry of Health Kenya, National AIDS Control Council, KEMRI, NCPD, The DHS Program II
      , 2015. Kenya demographic and Health Survey 2014. Available: https://dhsprogram.com/pubs/pdf/fr308/fr308.pdf
    5. 5.↵
      2. Paxton A ,
      3. Maine D ,
      4. Freedman L , et al
      . The evidence for emergency obstetric care. Int J Gynaecol Obstet 2005;88:181–93.doi:10.1016/j.ijgo.2004.11.026
      OpenUrlCrossRefPubMed
    6. 6.↵
      2. Yakoob MY ,
      3. Ali MA ,
      4. Ali MU , et al
      . The effect of providing skilled birth attendance and emergency obstetric care in preventing stillbirths. BMC Public Health 2011;11 S7.doi:10.1186/1471-2458-11-S3-S7
      OpenUrlCrossRefPubMed
    7. 7.↵
      2. Ni Bhuinneain GM ,
      3. McCarthy FP
      . A systematic review of essential obstetric and newborn care capacity building in rural sub-Saharan Africa. BJOG: Int J Obstet Gy 2015;122:174–82.doi:10.1111/1471-0528.13218
      OpenUrl
    8. 8.↵
      2. Ameh CA ,
      3. van den Broek N
      . Making it happen: training health-care providers in emergency obstetric and newborn care. Best Pract Res Clin Obstet Gynaecol 2015;29:1077–91.doi:10.1016/j.bpobgyn.2015.03.019
      OpenUrlCrossRefPubMed
    9. 9.↵
      2. van Lonkhuijzen L ,
      3. Dijkman A ,
      4. van Roosmalen J , et al
      . A systematic review of the effectiveness of training in emergency obstetric care in low-resource environments. BJOG 2010;117:777–87.doi:10.1111/j.1471-0528.2010.02561.x
      OpenUrlCrossRefPubMed
    10. 10.↵
      2. Bergh AM ,
      3. Baloyi S ,
      4. Pattinson RC
      . What is the impact of multi-professional emergency obstetric and neonatal care training? Best Pract Res Clin Obstet Gynaecol 2015;29:1028–43.doi:10.1016/j.bpobgyn.2015.03.017
      OpenUrlCrossRefPubMed
    11. 11.↵
      2. Ameh CA ,
      3. White S ,
      4. Dickinson F , et al
      . Retention of knowledge and skills after emergency obstetric care training: a multi-country longitudinal study. PLoS One 2018;13:e0203606.doi:10.1371/journal.pone.0203606
    12. 12.↵
      2. Banke-Thomas A ,
      3. Madaj B ,
      4. Kumar S , et al
      . Assessing value-for-money in maternal and newborn health. BMJ Glob Health 2017;2:e000310.doi:10.1136/bmjgh-2017-000310
    13. 13.↵
      2. Banke-Thomas AO ,
      3. Madaj B ,
      4. Charles A , et al
      . Social return on investment (SROI) methodology to account for value for money of public health interventions: a systematic review. BMC Public Health 2015;15:582.doi:10.1186/s12889-015-1935-7
      OpenUrlCrossRefPubMed
    14. 14.↵
      2. Nicholls J ,
      3. Lawlor E ,
      4. Neitzert E
      . A guide to social return on investment. 2nd edn. Liverpool: The Cabinet Office, 2012.
    15. 15.↵
      2. Banke-Thomas A ,
      3. Wilson-Jones M ,
      4. Madaj B , et al
      . Economic evaluation of emergency obstetric care training: a systematic review. BMC Pregnancy Childbirth 2017;17:403.doi:10.1186/s12884-017-1586-z
      OpenUrl
    16. 16.↵
      2. Vogel I
      . Review of the Use of ‘Theory of Change’ in International Development. London, 2012.
    17. 17.↵
      1. DFID
      , 2012. Business case and intervention summary: maternal and neonatal health human resource capacity building. Available: http://iati.dfid.gov.uk/iati_documents/3717740.odt
    18. 18.↵
      2. Namazzi G ,
      3. N KS ,
      4. Peter W , et al
      . Stakeholder analysis for a maternal and newborn health project in eastern Uganda. BMC Pregnancy Childbirth 2013;13:58.doi:10.1186/1471-2393-13-58
      OpenUrl
    19. 19.↵
      2. Scholten P
      , 2016. Value game. Available: http://www.valuegame-online.org/
    20. 20.↵
      2. Lee CC ,
      3. Li D ,
      4. Arai S , et al
      . Ensuring cross-cultural equivalence in translation of research consents and clinical documents: a systematic process for translating English to Chinese. J Transcult Nurs 2009;20:77–82.doi:10.1177/1043659608325852
      OpenUrlCrossRefPubMed
    21. 21.↵
      2. Bryson JM
      . What to do when stakeholders matter. Public Management Review 2004;6:21–53.doi:10.1080/14719030410001675722
      OpenUrlCrossRef
    22. 22.↵
      2. Ritchie J ,
      3. Lewis J
      . Qualitative research practice: a guide for social science students and researchers. London: SAGE Publications, 2003.
    23. 23.↵
      2. Braun V ,
      3. Clarke V
      . Using thematic analysis in Psychology. Qual Res Psychol 2006;3:77–101.doi:10.1191/1478088706qp063oa
      OpenUrlCrossRef
    24. 24.↵
      2. James R
      . Overview of cost definitions and costing methods. Washington D.C, 2010.
    25. 25.↵
      1. OANDA
      , 2018. GBP to KES 1/1/2014 to 31/12/2015.. Available: https://www.oanda.com/fx-for-business/historical-rates
    26. 26.↵
      1. Central Bank of Kenya
      , 2018. Kenyan Shillings to STG pound foreign exchange rates. Available: https://www.centralbank.go.ke/forex/
    27. 27.↵
      2. Adler AJ ,
      3. Ronsmans C ,
      4. Calvert C , et al
      . Estimating the prevalence of obstetric fistula: a systematic review and meta-analysis. BMC Pregnancy Childbirth 2013;13:246.doi:10.1186/1471-2393-13-246
      OpenUrl
    28. 28.↵
      2. Ronsmans C ,
      3. Vanneste AM ,
      4. Chakraborty J , et al
      . Decline in maternal mortality in Matlab, Bangladesh: a cautionary tale. The Lancet 1997;350:1810–4.doi:10.1016/S0140-6736(97)08012-4
      OpenUrl
    29. 29.↵
      2. Osei I ,
      3. Garshong B ,
      4. Banahene G
      . Improving the Ghanaian safe motherhood programme. In: Evaluating the effectiveness of alternative training models and other performance improvement factors on the quality of maternal care and client outcomes. Accra & Washington D.C: Population Council Frontiers in Reproductive Health, 2005.
    30. 30.↵
      2. Walker D ,
      3. McDermott JM ,
      4. Fox-Rushby J , et al
      . An economic analysis of midwifery training programmes in South Kalimantan, Indonesia. Bull World Health Organ 2002;80:47–55.
      OpenUrlPubMed
    31. 31.↵
      2. Boulenger S ,
      3. Dmytraczenko T
      , 2007. Cost of family care international’s skilled care initiative in Kenya and Tanzania. Bethesda, MD, USA. Available: http://www.familycareintl.org/UserFiles/File/SCI Costing Analysis.pdf
    32. 32.↵
      2. Kruk ME ,
      3. Pereira C ,
      4. Vaz F , et al
      . Economic evaluation of surgically trained assistant medical officers in performing major obstetric surgery in Mozambique. Br J Obstet Gynaecol 2007;114:1253–60.doi:10.1111/j.1471-0528.2007.01443.x
      OpenUrlCrossRef
    33. 33.↵
      2. Manasyan A ,
      3. Chomba E ,
      4. McClure EM , et al
      . Cost-effectiveness of essential newborn care training in urban first-level facilities. Pediatrics 2011;127:e1176–81.doi:10.1542/peds.2010-2158
      OpenUrlAbstract/FREE Full Text
    34. 34.↵
      2. Blomquist GC ,
      3. Dickie M ,
      4. O’Conor RM
      . Willingness to pay for improving fatality risks and asthma symptoms: values for children and adults of all ages. Resour Energy Econ 2011;33:410–25.doi:10.1016/j.reseneeco.2010.05.004
      OpenUrl
    35. 35.↵
      2. Wodinski L ,
      3. Wanke M ,
      4. Khan F
      . Impact evaluation of the H.E.R. Pregnancy program – final summary report, 2013.
    36. 36.↵
      2. Goudet S ,
      3. Griffiths PL ,
      4. Wainaina CW , et al
      . Social value of a nutritional counselling and support program for breastfeeding in urban poor settings, Nairobi. BMC Public Health 2018;18:424.doi:10.1186/s12889-018-5334-8
      OpenUrl
    37. 37.↵
      2. Saisana M ,
      3. Saltelli A ,
      4. Tarantola S
      . Uncertainty and sensitivity analysis techniques as tools for the quality assessment of composite indicators. J R Stat Soc Ser A Stat Soc 2005;168:307–23.doi:10.1111/j.1467-985X.2005.00350.x
      OpenUrl
    38. 38.↵
      2. Larsen-Cooper E ,
      3. Bancroft E ,
      4. Rajagopal S , et al
      . Scale matters: a cost-outcome analysis of an m-Health intervention in Malawi. Telemed J E Health 2016;22:317–24.doi:10.1089/tmj.2015.0060
      OpenUrl
    39. 39.↵
      2. Mekbib T ,
      3. Kassaye E ,
      4. Getachew A , et al
      . The FIGO save the mothers Initiative: the Ethiopia-Sweden collaboration. Int J Gynaecol Obstet 2003;81:93–102.doi:10.1016/S0020-7292(03)00071-7
      OpenUrlCrossRefPubMed
    40. 40.↵
      2. Chukudebelu W ,
      3. Ikeme A ,
      4. Okaro J , et al
      . Involving the private sector in improving obstetric care, Anambra state, Nigeria. The Enugu PMM team. Int J Gynaecol Obstet 1997;59 S107–S112.doi:10.1016/S0020-7292(97)00154-9
      OpenUrl
    41. 41.↵
      2. Crofts JF ,
      3. Mukuli T ,
      4. Murove BT , et al
      . Onsite training of doctors, midwives and nurses in obstetric emergencies, Zimbabwe. Bull World Health Organ 2015;93:347–51.doi:10.2471/BLT.14.145532
      OpenUrl
    42. 42.↵
      2. Vian T
      . Benefits and drawbacks of per diems: do allowances distort good governance in the health sector? Bergen, 2009.
    43. 43.↵
      2. Ridde V
      . Per diems undermine health interventions, systems and research in Africa: burying our heads in the sand. Trop Med Int Heal 2010;15:E1–E4.doi:10.1111/tmi.2607
      OpenUrl
    44. 44.↵
      2. Banke-Thomas A
      . What about the issues in using social return on investment as an evaluation tool? Eva J of Australasia 2018;18:64–8.doi:10.1177/1035719X17753984
      OpenUrl

    Footnotes

    • Handling editor Seye Abimbola

    • Contributors AB-T, BM and NvdB were involved in the initial conceptualisation of the review. AB-T and BM conducted the data collection. AB-T led the analysis, which was overseen by BM and NvdB. All authors were involved in writing the manuscript and approved the final version.

    • Funding Department for International Development (DFID/UKAid) for the ‘Making it Happen’ programme (Grant number 202945-101).

    • Disclaimer DFID/UKAid had no role in the design of the study, in collection, analysis and interpretation of data or in writing the manuscript.

    • Competing interests None declared.

    • Patient consent for publication Not required.

    • Ethics approval Ethical approval was received from the Research and Ethics Committee of the Liverpool School of Tropical Medicine (LSTM) (14.054) and the Kenyatta National Hospital Ethics and Research Committee (P718/12/2014).

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data sharing statement No additional data are available.