Article Text
PDF
Abstract
Introduction Kilkari is the largest maternal messaging programme of its kind globally. Between its initiation in 2012 in Bihar and its transition to the government in 2019, Kilkari was scaled to 13 states across India and reached over 10 million new and expectant mothers and their families. This study aims to determine the cost-effectiveness of exposure to Kilkari as compared with no exposure across 13 states in India.
Methods The study was conducted from a programme perspective using an analytic time horizon aligned with national scale-up efforts from December 2014 to April 2019. Economic costs were derived from the financial records of implementing partners. Data on incremental changes in the practice of reproductive maternal newborn and child health (RMNCH) outcomes were drawn from an individually randomised controlled trial in Madhya Pradesh and inputted into the Lives Saved Tool to yield estimates of maternal and child lives saved. One-way and probabilistic sensitivity analyses were carried out to assess uncertainty.
Results Inflation adjusted programme costs were US$8.4 million for the period of December 2014–April 2019, corresponding to an average cost of US$264 298 per year of implementation in each state. An estimated 13 842 lives were saved across 13 states, 96% among children and 4% among mothers. The cost per life saved ranged by year of implementation and with the addition of new states from US$392 ($385–$393) to US$953 ($889–$1092). Key drivers included call costs and incremental changes in coverage for key RMNCH practices.
Conclusion Kilkari is highly cost-effective using a threshold of India’s national gross domestic product of US$1998. Study findings provide important evidence on the cost-effectiveness of a national maternal messaging programme in India.
Trial registration NCT03576157.
- Health economics
- Maternal health
- Randomised control trial
- Prevention strategies
- Child health
Data availability statement
Data are available upon reasonable request. Data are available upon reasonable request to the study PI Dr. Amnesty LeFevre aelefevre@gmail.com.
This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.
Statistics from Altmetric.com
What is already known on this topic
There is a paucity of evidence on the value for money of digital health programmes, including direct to beneficiary solutions which provide mobile health information content directly to pregnant and postpartum women.
What this study adds
This study suggests that the Kilkari programme saved an estimated 13 842 lives across 13 states from December 2014 to April 2019 at a total programme cost of US$8.4 million.
The cost per live saved from US$392 ($385–$393) to $953 ($889–1092) well under the gross domestic product of US$1998 for India. Key drivers included call costs and incremental changes in coverage for key reproductive maternal newborn and child health practices.
How this study might affect research, practice or policy
This is the first study of its kind to demonstrate the cost-effectiveness of a direct to beneficiary mobile health programme being implemented at scale, under real-world conditions.
Introduction
Direct to beneficiary mobile health services which send health information to new and expectant mothers are among the few types of digital health programmes to have scaled widely in a range of countries globally. At least four programmes globally have scaled to reach over a million subscribers including Aponjon in Bangladesh,1 2 mMitra3 and Kilkari in India4 and MomConnect in South Africa.5 While evidence on the impact of these programmes is emerging,3 4 6 the limited available data on the value for money and affordability remains a critical impediment to transitioning from donor to government funding and enabling longer term sustainability.7
Kilkari is an outbound service that makes weekly, stage-based, prerecorded calls about reproductive, maternal, neonatal and child health directly to families’ mobile phones, starting from the second trimester of pregnancy and until the child is 1-year old. Between its inception in 2012 in Bihar, and its transition to the governmet in April 2019, Kilkari was scaled to 10 million subscribers in over 13 states across India. Current estimates of programmatic reach suggest that the programme has reached over 29 million women and their families across 18 states and currently has 2.5 million active users.8 Emerging evidence on the impact of Kilkari suggests that exposure to health information messages may increase immunisation coverage at 10 weeks and lead to shift in contraceptive methods—increasing the use of modern reversible contraceptive methods overall and slightly decreasing the proportion of men or women sterilised since the birth of the child.9 Evidence on the cost-effectiveness of the Kilkari programme, however, remains outstanding.
Evidence reviews of the cost-effectiveness and cost-utility of digital health solutions is emerging for mobile health solutions which target older adults,10 behaviour change communication apps,11 telemedicine in Asia12 and mHealth solutions more broadly.13 However, less is known about the value for money of direct to beneficiary mobile health programmes operating at scale.14 While these services have been implemented widely in a range of settings, the rapid pace of their scale-up has occurred without robust evidence generation on their impact or value for money.15 A model-based analysis of the Mobile Alliance for Maternal Action (MAMA) project in South Africa sought to forecast the costs and consequences of scaling-up the text-based delivery of health information messages to pregnant and postpartum women across the province of Gauteng in South Africa.14 An earlier study sought to explore the cost-effectiveness of Mobile Technology for Community Health (MOTECH) in Ghana—an interactive voice response (IVR) service similar to Kilkari but inclusive of an additional facility-based data capture application for frontline health workers.16 These studies have collectively suggested that direct to beneficiary mobile health information messages may be cost-effective; however, limitations in the underlying study design (none was conducted as part of randomised controlled trials (RCTs)) coupled with programmatic variations and differences in the scale of implementation, limits comparability and syntheses.
The goal of this study is to bolster evidence on the value for money of direct to beneficiary mobile health interventions in low-income and middle-income countries where the majority of maternal and child deaths occur each year. This study aims to determine the cost-effectiveness of exposure to Kilkari—one of the world’s largest direct to beneficiary mobile health service—as compared with the status quo of no mobile health information messages across 13 states in India. We start by presenting programme costs associated with the gradual start-up and implementation of programme activities by state. We next present modelled estimates of lives saved based on data drawn from an individually RCT in the central Indian state of Madhya Pradesh.17 Finally, we present estimates of cost-effectiveness and findings from uncertainty analyses. This is the first study of its kind to explore the value for money of a direct to beneficiary mobile health programme at scale under real-world conditions of implementation.
Methods
Study setting
India is home to over 1.3 billion people disbursed across 28 states and eight union territories. The Kilkari programme was designed and piloted by BBC Media Action in the Indian state of Bihar in 2013, and then redesigned and scaled across 13 states in collaboration with the Ministry of Health and Family Welfare (MOHFW) between 2015 and 2019 (figure 1).
Kilkari program launch dates by State. Colours denote the year of launch for each state: green for 2015, red for 2017 and blue for 2018.
Study population
This study draws on data from two populations of pregnant and postpartum women: (1) Kilkari subscribers in the 13 states where programme implementation was underway as part of the Kilkari national programme supported by BBC Media Action and the MOHFW and (2) women enrolled into an individually RCT in four districts of the central Indian state of Madhya Pradesh. The former are pregnant and postpartum women subscribed to Kilkari based on the mobile number captured in governmental tracking registries called, depending on the state, the Maternal and Child Health Tracking System or Reproductive and Child Health system. Online supplemental table 3 summarises the total population of pregnant women eligible for Kilkari across 13 States. An average of 21% of pregnancies across 13 states was recorded in government tracking registries and, thus, eligible for Kilkari.18 Primary data on the demographic profile and health behaviours of Kilkari subscribers are not available across the 13 states where implementation in underway. Accordingly, data on the Kilkari programme’s impact on health outcomes were generalised from an RCT of Kilkari conducted from 2017 to 2020 in the central Indian state of Madhya Pradesh.19 This included data on beneficiary reported care-seeking practices and health behaviours. RCT participants (n=5095) were women of 12–34 weeks of gestation at time of enrolment, more than 18years of age, who could speak and understand Hindi, and reported owning or having access to a mobile phone during the day when Kilkari calls were likely to come. These women were identified during a household listing survey described in detail elsewhere.9 17
Supplemental material
Comparators
This study compared women randomised to receive health information messages as part of the Kilkari programme against a status quo of no messages.
Kilkari is free of cost to subscribers. Depending on the timing of enrolment, subscribers may receive up to 72 weekly, stage-based, prerecorded calls about reproductive, maternal, neonatal and child health directly to families’ mobile phones, starting as early as the second trimester of pregnancy and ending when the child is 1-year old.4 Across health content areas, 18% of cumulative call content is on family planning (benefits of family planning, modern reversible methods, sterilisation, pregnancy tests); 13% on child immunisations (diseases covered, doses); 13% on nutrition (malnutrition, growth monitoring, maternal anaemia); 12% on infant feeding (quality of food, breastfeeding, complementary feeding, child anaemia); 10% on pregnancy care (antenatal care, institutional delivery, rest, tetanus toxoid, emergency services); 7% on entitlements; 7% on diarrhoea; 7% on postnatal care (newborn danger signs, cord care, hypothermia) and the remainder on a range of topics including intrapartum care, water and sanitation and early childhood development. Additional details on the programme are reported elsewhere.9 17
Perspective and analytic time horizon
Data were collected from a programme perspective for the analytic time horizon of October 2015 to April 2019. The programme perspective includes all costs incurred by the implementing partners in the design and implementation of the Kilkari programme. The programme perspective was selected because it most closely aligns to the costs future payers (Government of India, external donors) would likely incur to introduce and support continued programme implementation. The programme is not anticipated to have resulted in beneficiaries incurring costs to receive calls, nor to the health system since the service is provided directly to the mobile phones of those subscribed drawing from existing government tracking registries. The time horizon used corresponds to the window of time BBC Media Action was supporting the national scale-up of Kilkari. Incremental cost-effectiveness ratios are presented for calendar years 2016–2018 each of which include a full 12 months of programmatic activities.
Costs
Economic costs were estimated based on financial records maintained by implementing partners including BBC Media Action, the Grameen Foundation, Dimagi and Beehyv. Costs are categorised into capital and recurrent costs and presented for the core ‘ingredients’ or activities which comprise Kilkari. Capital costs included one-time costs associated with infrastructure (third-party hardware and software, hosting telecommunications infrastructure), technology (software licensing, MOTECH engine costs, and IVR professional services fee), content creation and training. Recurrent costs included telecommunication call costs, data centre and technical support, personnel (BBC Media Action, Program Management Unit, management and operations), office space and other miscellaneous costs. Costs were adjusted into 2019 base year US dollars (coinciding with the final year of effect estimates) using local consumer price indices and market exchange rates. Capital costs were annualised over the lifespan of the project using a 3% discount rate.
Outcomes (selection, measurement, valuation)
Maternal and child (0–12 months) lives saved were the primary the health outcome. Lives saved were derived using the Lives Saved Tool (LiST) which ‘calculates changes in cause-specific mortality based on intervention coverage change, intervention effectiveness for that cause and the percentage of cause-specific mortality sensitive to that intervention’.20 Details on the methods underpinning LiST are outlined in detail elsewhere.20 The number of lives saved was estimated for each of the 13 states where Kilkari implementation is underway. Estimates of the total population of women eligible for Kilkari were inputted into LiST18 along with data on incremental changes in coverage drawn from the Kilkari RCT in Madhya Pradesh.19 Coverage estimates were used for only those health behaviours observed to have a statistically significant difference across RCT study arms including modern reversible contraceptive method use, sterilisation and immunisations at 10 weeks. These findings and the underpinning methods used to derive them are presented in detail in a companion paper published elsewhere.4 6 19 Online supplemental tables 1–2 presents coverage input estimates used in LiST. Multiple iterations of LiST were run for each state to generate upper and lower bound estimates of lives saved using the 95% CI) around point estimates of coverage for each behaviour.
Study parameters
Table 1 presents parameters for the 2018 calendar year. Online supplemental tables 3–4 present parameters for 2016 and 2017. High and low estimates for costs are based on a ±10% change around each parameter. For health effects, the upper and lower bounds of the 95% CIs for coverage estimates were used in LiST to generate high and low scenarios.
Parameters for 2018 Kilkari programme costs and effects for implementation across 13 states
Analytics
Analyses used to estimate differences in coverage for target health behaviours are described in depth elsewhere.19 In brief, to assess exposure to Kilkari content, call data records from the IVR system were linked to baseline and endline survey data. Listening patterns were assessed for each subscriber by call, for the duration of their subscription to Kilkari using call data records. The latter provides evidence on subscriber engagement with calls, including the duration of listening to individual calls. To link individual call listening patterns to health outcomes, we mapped the content of calls to key outcome indicators measured in household surveys. Exposure was defined at a listening threshold of 50% or more of the cumulative duration of the calls mapped to the outcome. Primary analyses of outcomes were done with modified intention-to-treat (ITT) analyses at the individual level, so that outcomes were analysed regardless of the degree of listening to Kilkari. To assess the impact of exposure on outcomes, compliance-adjusted treatment effects (CATE) were additionally generated using the instrumental variable methodology.4 6 Coverage estimates were inputted into LiST and scenarios run for each state which considered the duration of implementation, total fertility rate and Kilkari-adjusted population coverage. Base case estimates for lives saved are based on ITT estimates of coverage; arguably the most conservative approach.
To characterise heterogeneity, we additionally estimated the number of lives saved across socioeconomic strata, and similarly, based on Kilkari exposure, drawing from CATE findings. Socioeconomic strata were derived using principal components analyses. Cost data by subgroup were not available and, therefore, estimates of cost-effectiveness by subgroup not generated.
Statistical analyses were carried out in R21 and Microsoft excel (Microsoft, Redmond, Washington). The impact of uncertainty was assessed through probabilistic sensitivity analysis, using standard Monte-Carlo simulation resampling.22 In this method, data points were randomly sampled from original data, with replacement, and ICERs were calculated. This process was repeated to represent what results might arise if a large number of similar trials were performed. These calculations were performed in Excel using a Visual Basic-based macro to perform the resample automatically. In total, 1000 iterations were generated for each simulation and plotted on two-dimensional cost-effectiveness planes. Results were presented as cost-effectiveness acceptability curves, which are standardised tools for summarising the probability of cost-effectiveness based on variations in the ceiling ratio.23 We additionally evaluated findings against a ceiling ratio based on the per capita gross domestic product (GDP) for India in 2018 (US$1998)24; a threshold favoured by Commission on Macroeconomics and Health for assessing cost-utility analyses using disability-adjusted life years (DALYs) as the primary outcome. In the absence of a similar standard for cost-effectiveness analyses25 which use lives saved as the primary outcome, we present this threshold as a conservative proxy in addition to the cost-effectiveness acceptability curves, which allow users to weigh results against a range of alternative willingness to pay thresholds.
Role of the funding source
The Bill and Melinda Gates Foundation had no role in the study design; collection, analysis and interpretation of data; in the writing of the report or in the decision to submit the paper for publication. All authors confirm that they had full access to all the data in the study and accept responsibility for the publication submitted.
Patient and public involvement
This is a secondary analysis which draws on primary data collected as part of the Kilkari RCT in Madhya Pradesh. As part of the RCT, beneficiaries were engaged during household surveys and qualitative interviews. The latter included engaging a small number of patients in the refinement of survey tools. Unfortunately because of COVID-19 and associated travel restrictions, patients could not be involved in the dissemination of study findings. However, public dissemination of the results has occurred through a number of presentations in India and elsewhere globally.
Results
Summary of main results
Table 2 summarises programme costs for the 2015–2019 window. Overall, capital costs were an average of 12% of total programme costs, while recurrent costs comprised 88%. Kilkari call costs constituted 23% of total costs, followed by costs associated with BBC Media Action personnel (22%), the programme management unit (15%) and other technical support (12%). Among capital costs, infrastructure (6% of total costs) and IVR licensing and professional services (2%) were the leading cost drivers.
Total programme costs (USD) incurred to support implementation and expansion to 13 states from 2015 to 2019
Table 3 summarises total lives saved by state and year, adjusted for duration of implementation. Six states (Jharkhand, Madhya Pradesh, Odisha, Rajasthan, Uttarakhand and Uttar Pradesh) had over 1200 days of implementation and comprised the largest proportion of lives saved. One-fifth (25%) of total lives saved came from Uttar Pradesh, followed by Rajasthan (20%) and Madhya Pradesh (15%). The Indian state of West Bengal launched in February of 2018 and as a result, had the lowest (4%) overall proportion of lives saved. The majority of lives saved (96%) occurred in children 0–12 months with the remaining (4%) attributed to maternal lives saved.
LiST estimates of the number of lives saved by State 2015–2019
Table 4 summarises incremental cost-effectiveness ratios for lives saved for 2016–2018. Results suggest that the cost-effectiveness of Kilkari improved with time and scale. By 2018, when implementation was underway in 13 states, the average incremental cost per live saved was US$391.78 ($384.84–$393.30) as compared with US$953.29 ($889.21–$1091.56) in 2016 when implementation was only newly underway in seven states.
Incremental cost-effectiveness ratios 2016–2018
Effect of uncertainty
Figure 2 presents a tornado diagram for 2018 costs and lives saved. The leading drivers of cost-effectiveness were technology and call costs, followed by programme personnel costs to manage and support the programme. Online supplemental figure 1 depicts the cost-effectiveness plane, while figure 3 presents cost-effectiveness acceptability curves for 2016–2018. With a cost per live saved of US$391.78 in 2018, Kilkari falls beneath the World Bank’s GDP per capita threshold of US$1998 for 2018.
Supplemental material
Tornado diagram of 2018 costs and lives saved.
Willingness to pay (WTP) to avert maternal and child mortality.
Discussion
Study findings present evidence on the value for money of Kilkari—one of the world’s largest direct to beneficiary mobile health service. This study is the first of its kind conducted of a digital health programme being implemented at scale in India and elsewhere globally. Findings suggest that an estimated 13 842 lives were saved across 13 states from October 2015 to April 2019; 25% of these are estimated to have occurred in the Indian state of Uttar Pradesh. From October 2015 to April 2019, nearly US$8.4 million was spent to support the introduction and ongoing implementation of Kilkari across 13 states. IVR call costs were the leading driver of costs (23%). The incremental cost per life saved ranged from US$953 in 2016 to US$392 in 2018. The incremental cost-effectiveness ratio decreased over time and with increasing scale. Based on these findings, Kilkari is highly cost-effective using a GDP threshold and compares favourably with other low-cost high priority interventions.26
Estimates on the number of incremental lives saved were derived using LiST. LiST is a mathematical modelling tool which allows users to estimate the impact of changes in coverage for reproductive maternal newborn and child health interventions on mortality in low and middle-income countries.20 LiST has been featured in over 150 peer review publications27 and used to model the impact interventions which may have on mortality in a range of settings globally including Afghanistan, Bangladesh,28 Malawi,29 Mozambique, Niger30 and South Africa. In the context digital health solutions, it has been used in economic evaluations to measure the number of lives saved as result of a frontline health worker application in Bangladesh28 and direct to beneficiary solutions in Ghana16 and South Africa.14 The external validity of LiST has been assessed through comparisons with alternative data sources including measured data from vector control studies,31 demographic and health survey data32 and community-based intervention trials.33 The external validity of LiST with regards to the estimated impact of changes in coverage of reproductive health interventions has not yet been assessed to our knowledge. While this absence of more comprehensive data on LiST’s external validity is a limitation, others have sought to highlight the importance such tools may nevertheless have particularly in modelling effects that RCTs cannot reasonably be expected to detect.34 In the context of interventions like Kilkari, the impact on mortality is likely to fall under 5%. Detecting such a small margin of change would be cost prohibitive and simply infeasible given larger global funding challenges for digital health programmes and their evaluations.
Table 5 presents a league table contextualising findings with estimates drawn from the literature. In the absence of comparable data using lives saved as an outcome, we have relied on comparisons with publications which present data on the cost per DALYs averted. DALYs are a summary measure of overall disease burden expressed in terms of years lost due to ill-health, disability or early death.35 In the absence of disability weights for the full range of health conditions considered in our analyses,36 we have not presented DALYs as an outcome measure. Nevertheless, comparisons with our findings may provide some insights into how Kilkari compares with alternative resource uses.
League table
Comparisons of results observed with other digital health programmes are challenging given the limited number of studies present in the literature, variations in the programmes being evaluated and heterogeneity in the methodological approaches undertaken. Among the other large-scale direct to beneficiary mobile health solutions, evidence on value for money was available only for the MAMA project in South Africa.14 MAMA was one of the precursor programmes to MomConnect5 15—the National Department of Health’s flagship mobile messaging programme which sent up to 140 text messages to new and expectant women attending pregnancy services in the public sector.15 Efforts to determine the cost-effectiveness of MAMA were conducted as part of a retrospective study in six health centres in Gauteng province and sought to gradually model the implications in terms of costs and effects of scaling-up services to pregnant women throughout Gauteng province, South Africa from 2012 to 2017.14 Results suggested that the incremental costs per live from a societal perspective ranged from US$56 011 in year 1 of implementation to US$5652 in the fifth year.14 Findings from the MAMA study compare to those observed in our study (cost per live saved of US $392-$953) and suggest that with increasing population-level coverage and expansion, the programme’s cost-effectiveness improves. Similar to findings in our study, the leading drivers of cost for MAMA were call costs (31%).14
Despite the comparability of findings, some methodological differences in the approach undertaken to assessing cost-effectiveness are noteworthy. First, economic costs in the MAMA study were estimated from a societal perspective inclusive of programme, health systems and user costs for a 5-year analytic time horizon (2012–2017). Collecting data from a societal perspective was not possible for Kilkari and instead a programme perspective was taken. In contrast to MAMA which was provided through government health facilities and depended on government providers to register subscribers, Kilkari was provided directly to beneficiaries and did not rely on added health system inputs including provider time to support registration. Instead, the subscriber population for Kilkari was drawn from secondary data within government tracking registries including the last menstrual period for women and their phone number. This programmatic strategy of enrolment based on government tracking registry data was designed to facilitate programme expansion and scalability, while limiting the burden placed on public sector providers. In terms of beneficiary costs, direct to beneficiary mobile health programmes may have cost implications for subscribers on two fronts: (1) bolstering utilisation of health services and in turn, costs for transport, wages lost, child care, etc and (2) costs to receive health information content, including hardware and phone credit. Both programmes were provided free of cost to beneficiaries. The MAMA study sought to attribute a monetary value to changes in health service utilisation, which was appropriate given that the health outcomes assessed were linked to pregnancy and immunisation care seeking in the public sector. In the context of Kilkari, limitations in the total interview time available to administer structured surveys to women enrolled into the RCT meant that questions on direct and indirect costs to beneficiaries for health careseeking were not possible to capture. Beyond the measurement of costs, health effects in both studies were derived via a similar approach of inputting coverage data into LiST and modelling lives saved. However, the quality of evidence differed markedly across studies. Kilkari data were drawn from a large (n=5095) individual RCT implemented across four districts in Madhya Pradesh, whereas MAMA impact estimates were drawn from a small-scale (<200 mother–infant pairs) retrospective case-control study conducted in six health clinics in Gauteng province.37 The MAMA study further sought to forecast the potential costs and consequences of programme expansion based on this limited primary data. In the context of Kilkari, primary data on costs of programme expansion across 13 states were used.
Limitations
Estimates of health effects were modelled using LiST and based on incremental changes in coverage observed as part of an RCT conducted in four districts of the central Indian state of Madhya Pradesh. While RCT findings provide the most definitive evidence to date on the impact of Kilkari, they nevertheless may not be representative of the programme’s impact in other contexts. India is extremely diverse and states such as Bihar and Uttar Pradesh tend to perform below Madhya Pradesh for most maternal and child health care seeking and health behaviours. State-level variations in mobile phone access and use, particularly among women, may too have implications for programme reach, exposure and impact.38 We further note that data on Kilkari’s impact on health outcomes related to family planning are based on self-reported use contraceptive methods, which may be subjected to social desirability bias. There additionally could be information and recall biases since these data were collected at a single timepoint (12 month’s postpartum) and yet pertain to practices occurring during the 12-month window preceding the interview. Cost estimates are presented from a programme perspective. Costs to beneficiaries include the cost of owning the handset and data, along with potential costs incurred in care seeking for target health behaviours. These costs were not possible to collect as part of the RCT in Madhya Pradesh due to constraints in the number of questions possible to ask within a limited interview window. Additional costs to the health system were similarly not captured but are likely minimal. These are likely to include frontline health worker time costs to collect and register the phone numbers for couples as well as costs associated with increases in care seeking as a result of exposure to health information messages, which might bolster beneficiary awareness. Frontline health workers register couples as part of routine health information systems data collected—programme activities have not led to a modification in this pre-existing behaviour, rather they simply leverage existing data as the sampling frame.
Conclusion
From 2016–2018, the 13-state implementation of the Kilkari programme was associated with a cost per live saved of US$392–US$953. These findings suggest that Kilkari is highly cost-effective and compares favourably with alternative resource uses for maternal and child health in India. This study contributes to the limited evidence base on the value for money of digital health solutions.
Supplemental material
Data availability statement
Data are available upon reasonable request. Data are available upon reasonable request to the study PI Dr. Amnesty LeFevre aelefevre@gmail.com.
Ethics statements
Patient consent for publication
Ethics approval
This study involves human participants and was approved by Institutional Review Boards from the Johns Hopkins Bloomberg School of Public Health in Baltimore, Maryland USA and Sigma Research and Consulting in Delhi, India provided ethical clearance for study activities. Verbal informed consent was obtained from all study participants. Participants gave informed consent to participate in the study before taking part.
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Handling editor Lei Si
Twitter @ac1112, @priydee
Collaborators The Kilkari Impact Evaluation Team* (listed in Alphabetical order) Smisha Agarwal, Salil Arora, Jean JH Bashingwa, Aarushi Bhatanagar, Sara Chamberlain, Rakesh Chandra, Arpita Chakraborty, Neha Dumke, Priyanka Dutt, Anna Godfrey, Suresh Gopalakrishnan, Nayan Kumar, Simone Honikman, Alain Labrique, Amnesty LeFevre, Jai Mendiratta, Molly Miller, Radharani Mitra, Diwakar Mohan, Deshen Moodley, Nicola Mulder, Angela Ng, Dilip Parida, Nehru Penugonda, Sai Rahul, Shiv Rajput, Neha Shah, Kerry Scott, Aashaka Shinde, Aaditya Singh, Nicki Tiffin, Osama Ummer, Rajani Ved, Falyn Weiss, Sonia Whitehead.
Contributors AEL is the guarantor of the paper content, the overall study PI, helped to secure the funding, led the design of the study tools, supported oversight of field work and analysis, and wrote the first draft of the manuscript. JM collected, helped to analyse and interpret costing data. YJ helped to conduct probabilistic sensitivity analyses, interpret the results, and edit the manuscript. MM and OU generate estimates of the number of lives saved with inputs from AEL and DM. NS support the design, implementation and analysis of survey data used to generate coverage estimates. KS helped to secure the funding, design the study tools, support the field work, input on analyses, and edit the manuscript. SC helped to secure the funding, draft and review study tools, interpret data analyses and study findings, and edit the manuscript. AC helped to draft and review study tools for effect measurement; support data collection, including training, monitoring; interpret data analyses and study findings, and edit the manuscript. AG and PD helped to draft and review study tools for the effect measurement, interpret data analyses and study findings, and edit the manuscript. DM helped to secure funding, helmed the study design including sampling and randomisation, helped draft study tools, led the data analysis and edited the manuscript.
Map disclaimer The inclusion of any map (including the depiction of any boundaries therein), or of any geographic or locational reference, does not imply the expression of any opinion whatsoever on the part of BMJ concerning the legal status of any country, territory, jurisdiction or area or of its authorities. Any such expression remains solely that of the relevant source and is not endorsed by BMJ. Maps are provided without any warranty of any kind, either express or implied.
Competing interests All authors have completed the Unified Competing Interest form (available on request from the corresponding author) and declare that the research reported was funded by the Bill and Melinda Gates Foundation. AG, SC, PD are employed by BBC Media Action; one of the entities supporting program implementation. The authors do not have other relationships and are not engaged in activities that could appear to have influenced the submitted work.
Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.