Article Text
PDF
Abstract
Kilkari is one of the largest maternal mobile messaging programmes in the world. It makes weekly prerecorded calls to new and expectant mothers and their families from the fourth month of pregnancy until 1-year post partum. The programme delivers reproductive, maternal, neonatal and child health information directly to subscribers’ phones. However, little is known about the reach of Kilkari among different subgroups in the population, or the differentiated benefits of the programme among these subgroups. In this analysis, we assess differentials in eligibility, enrolment, reach, exposure and impact across well-known proxies of socioeconomic position—that is, education, caste and wealth. Data are drawn from a randomised controlled trial (RCT) in Madhya Pradesh, India, including call data records from Kilkari subscribers in the RCT intervention arm, and the National Family Health Survey-4, 2015. The analysis identifies that disparities in household phone ownership and women’s access to phones create inequities in the population eligible to receive Kilkari, and that among enrolled Kilkari subscribers, marginalised caste groups and those without education are under-represented. An analysis of who is left behind by such interventions and how to reach those groups through alternative communication channels and platforms should be undertaken at the intervention design phase to set reasonable expectations of impact. Results suggest that exposure to Kilkari has improved levels of some health behaviours across marginalised groups but has not completely closed pre-existing gaps in indicators such as wealth and education.
- Epidemiology
- Maternal health
- Public Health
Data availability statement
The RCT data are available upon request from Diwakar Mohan (corresponding author) and Amnesty LeFevre (the study PI). NFHS-4 data is available for download from the Demographic and Health Surveys website (https://dhsprogram.com/data/available-datasets.cfm).
This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.
Statistics from Altmetric.com
Summary box
By necessity, ‘direct to beneficiary’ mobile health programmes target those with phone access and thus exclude the poorest and most marginalised.
Understanding inequities in who participates and benefits from mobile health programmes is vital for designing strategies to reach the most marginalised.
Exposure to Kilkari appears to benefit the higher educated among the poorest in terms of wealth quintiles.
Therefore, education may be a key enabler to help improve health behaviours among those exposed to Kilkari content.
The way forward for digital ‘direct to beneficiary’ communication programmes may be to customise and target health information based on the characteristics of specific segments of the population.
Introduction
There is much interest in ‘direct to beneficiary’ digital health communications programmes as a potential mechanism for addressing critical gaps in access to and receipt of health information, particularly in low-income and middle-income countries.1–10 Maternal mobile messaging programmes are among the few examples of such programmes that have scaled widely in different settings.11 12 Yet, to reach target women with health information, such programmes usually require beneficiaries to have access to a mobile phone as a prerequisite for participation. While overall rates of phone ownership are increasing globally, disparities in household ownership, and, more specifically, women’s access to and use of phones, persist.13–15 In India, despite near universal phone access among men, only half of women have access to mobile phones.16 Women with access to phones have poorer quality phones then men and frequently find their use constrained by a range of factors from phone sharing practices to social norms around securing credit.17 18
Kilkari is the one of the largest maternal mobile messaging programmes in the world, available to subscribers free of charge. Established in 2013 in Bihar, India, Kilkari had scaled to 13 states to reach over 10 million subscribers by April 2019. Subscribers received up to 90 min of content via up to 72 weekly calls from the fourth month of pregnancy until the child is 1 year old.
Understanding inequities in who participates and benefits through the life of a programme is vital for designing strategies to reach the most marginalised, and to optimise impact. Despite emerging evidence linking maternal messaging programmes to changes in health outcomes, little is known about the differential effects of these programmes across key population subgroups, including the least educated, poorest and most ethnically disadvantaged.7 12 19–23
In table 1, we present a framework for assessing equity in ‘direct to beneficiary’ health programmes by adapting the Tanahashi healthcare access framework and applying it to Kilkari in India.24 Data were drawn from three sources: an endline survey of women participants who received the Kilkari intervention as part of a randomised controlled trial (RCT) in Madhya Pradesh, call data records from Kilkari subscribers in the RCT sample and data on household mobile phone ownership from the Indian National Family Health Survey (NFHS-4) India from 2017 to 2020.25 26 A detailed description of the data sources and analytical methods are provided in box 1.
Description of data sources and analytical methods
As part of efforts to determine the impact of Kilkari on key reproductive maternal newborn and child health behaviours, an individually randomised controlled trial (RCT) was conducted in four districts of Madhya Pradesh, India from 2017 to 2020.25 26 This analysis explores programme eligibility, enrolment, reach, exposure and affect across three dimensions of equity—education, caste and wealth. The eligibility and enrolment were based on comparisons to the National Family Health survey-4 data which surveyed households and women of reproductive age all across India. The questions considered for the analysis include whether a household lists a mobile phone in its assets (household ownership) and if women report having a mobile phone they can use (women’s access to mobile phones). The health outcomes (affect) considered in this analysis is based on self-reports at endline by women in the trial who received the intervention. The outcomes included standard indicators from the maternal and child continuum of care such place of childbirth, infant feeding and postpartum contraceptive practices. The subscribers’ reach and exposure to the intervention was assessed using call records stored by the programme database with data on the status (answered or not) and the duration of the call. Years of schooling completed was used to create four categories based on educational level—no schooling (0 years), primary (1–5 years), secondary (6–10 years) and higher (10 or more years). Caste categories were based on the classification established by the Government of India—that is, scheduled castes, schedule tribes, other backward classes and the general caste category. Wealth categories were based on dividing the survey sample into quintiles using a score created by a principal components analysis of household assets.37 Survey analyses were carried out with R V.3.4 and graphics generated with ggplot2 package.38 Summary measures of equity were produced using the WHO HEAT PLUS application, designed to help equity analysis using disaggregated estimates from various data sources.39–41 To assess inequities in health outcomes among those enrolled, we looked at subscribers exposed and not exposed to Kilkari in the intervention arm of the RCT using two summary measures of equity1: Slope of Index Inequality (SII) and2 Relative Concentration Index (RCI). The SII is an absolute weighted measure of inequity which provides an overall estimate of changes in health behaviour across the dimensions of equity assessed—wealth, caste and education in our case. Bounded between the values of −100 and +100 (if multiplied by 100), the SII indicates the absolute difference in estimated values of an indicator between the most-advantaged and most-disadvantaged using a regression model. In contrast, the RCI is a relative measure which reflects proportional inequality and thus an understanding of where one group stands in relation to other groups.41 42 Both SII and RCI take the value zero if there is no inequality with higher values indicating greater inequality. Positive values indicate a concentration among the advantaged, while negative values indicate a concentration among the disadvantaged.41 43
Mapping the concepts of reach and coverage of digital programmes to the Tanahashi framework
Who benefits from the type of information provided by programmes like Kilkari?
In theory, stage appropriate reproductive, maternal, neonatal and child health (RMNCH) information is expected to benefit all households with pregnant women and new mothers with infants under 1 year of age. Across the 13 states where Kilkari implementation is ongoing, an estimated 50 million women become pregnant every year.27 Data on the sociodemographic characteristics of those who have a ‘need for information’ among this population are limited. However, 2015 NFHS data indicate that total fertility rates are highest among the poorest and among those with no schooling, which may suggest greater representation of marginalised groups among pregnant and postpartum women than in the broader population at any given time.28 While it is difficult to quantify which subgroups have the most ‘need for information’, it would not be unreasonable to assume that the less educated, poorest and most marginalised castes are likely to benefit from such an intervention.29–31
Who is eligible to receive Kilkari?
To be eligible to receive Kilkari in 13 Indian states, women must be pregnant and beyond the twelfth week of gestation, or have an infant less than a year old and, most importantly, must be able to provide their health worker with a mobile phone number (personal or otherwise) to receive prerecorded weekly calls. There are significant differences in household level ownership of mobile phones across key sociodemographic characteristics, and these inequities are more pronounced for women’s reported access to mobile phones.16 Data from 2015 NFHS indicates that the gap in household mobile phone ownership between the advantaged general caste (GC) and least advantaged scheduled tribe (ST) is 15.4%, while the gap in women’s access to mobile phones between the GC and ST is 27.1%.16 These differences in household ownership of phones and women’s reported phone access were different in magnitude between the poorest and richest (21% and 47%) segments of the population, and those with higher education and no schooling (13.9% and 55.3%).16 Overall differentials across these sociodemographic characteristics in women’s phone access contributed to the introduction of inequities in the population eligible to receive Kilkari. These inequalities need to be considered by programme implementers designing digital ‘direct to beneficiary’ health communication interventions so that reasonable expectations of programme reach, and impact can be set, and to prevent the exacerbation of inequities.
Kilkari was originally designed as part of an integrated social and behavioural change communication programme in the state of Bihar to strengthen RMNCH practices and generate demand for public health services. Communication was layered through different channels, including face-to-face communication by front-line health workers, to increase reach and exposure and improve diffusion of information.32 While several interventions were adopted by other state governments, and nationally, they were scaled independently, according to different timelines, which rarely overlapped. Other interventions were not scaled at all, creating gaps in the programme’s theory of change.33
Who is subscribed (enrolled) to Kilkari?
While the sociodemographic profile of those subscribed to Kilkari at scale is not available, the RCT sample in Madhya Pradesh, where respondents were enrolled directly at the community level by study researchers, provides information on characteristics such as wealth, caste and education.25 Figure 1 depicts differences in the distribution of key characteristics for women with and without phone access in rural Madhya Pradesh based on the NFHS population level survey, as compared with the women who were enrolled in the intervention arm of the Kilkari RCT, who reported having access to a phone between the hours of 7:00 and 20:00. The restricted time criteria were required to comply with the Telecommunications Regulatory Authority of India’s directive that automated prerecorded outbound calls must be made within socially acceptable hours. A comparison of the NFHS and RCT samples (figure 1) conveys two key points1: the RCT sample is similar in profile to the population-based NHFS sample of women 15–49 years of age with access to a phone (denoted by the grey vs red bars), and2 there are stark differences in educational levels and caste between those women enrolled in the Kilkari RCT and those without access to a mobile phone according to the NFHS survey (grey vs blue bars). These differences underscore the higher educational level of those enrolled in even the most basic of ‘direct to beneficiary’ mobile health communication programmes—that is, programmes that can be accessed from the most basic mobile phone with no additional software or skill beyond answering a phone call.
Caste and education of enrolled participants in the intervention arm of the Kilkari randomised controlled trial (RCT) as compared with their distribution among women reporting phone access from National Family Health Survey (NFHS-4). OBC, other backward classes.
Who is reached by Kilkari?
Once enrolled in Kilkari, subscribers can receive up to 72 weeks of calls starting in the fourth month of pregnancy until the child is a year old. Calls are timed to match the gestational age at enrollment and therefore, the number of calls per subscriber may vary in accordance with the timing of entry into the programme. Subscribers are ‘reached’ if the Kilkari call is ‘successful’ —that is, the call not only reaches the handset but is answered. To optimise reach, Kilkari attempts to call the same subscribed mobile phone numbers up to nine times each week—three times the first day, and two times over the next 3 days—to reach a target subscriber. Subscribers are given the option to unsubscribe from the service at any time. Although there are no significant differences in ‘successful’ call rates by caste, those with no schooling and the poorest are the least likely to answer at least one Kilkari call. One caveat is that a successful call pick up does not guarantee that the messages were picked up by the target subscriber, especially in large rural households, phone calls are picked up by children or other family members.
Figure 2 illustrates the number of call attempts it takes to reach a subscriber among the richest and poorest 20% of the RCT intervention arm sample, by wealth. On average, 60% of subscribers from the richest quintile have been reached by the fifth call attempt as compared only 40% of those from the poorest quintile. The number of attempts needed to successfully reach a subscriber may vary based on a range of factors including sociodemographic characteristics, phone characteristics (available credit, functionality), environmental factors (available electricity, location of top-up shops) and the social norms underpinning phone access and use.
Number of call attempts made to successful reach a Kilkari subscriber for the richest and poorest quintiles of subscribers in the intervention arm of RCT. RCT, randomised controlled trial.
Who is exposed to Kilkari content?
Kilkari calls range from 60 to 108 s in length. Calls are comprised of three parts: introduction to the topic (20% of the length of an individual call), core content (60%) and closing (20%). In this study, to be considered ‘exposed’, subscribers needed to listen to 50% or more of the cumulative content they were eligible to receive. This definition is different from that used in the impact paper, which instead considered exposure to specific content about specific health practices to assess causality.25
Overall, those in the most marginalised groups in the intervention arm of the Kilkari RCT sample—that is, those in scheduled castes and ST, those with no education and the poorest, were less likely to be exposed to Kilkari (figure 3). This could be due to factors including more frequent changes in mobile phone numbers (SIM churn) among these groups, and these groups being less able to answer calls during the day, less able to maintain the battery charge of their phones, less able to retain sufficient phone credit to receive calls and more likely to live in areas with poor network connectivity. While no subscribers were deactivated by the RCT for low listening or not answering calls, in the scaled Kilkari programme in 13 states, SIM churn could result in subscribers not receiving calls and, hence, being deactivated to free up infrastructure for Kilkari subscribers who were able to answer calls. This could result in marginalised subgroups being more likely to be deactivated by the scaled programme, while those staying on until the end of the programme might be those with better access to phones and more reliable network connectivity.
Characteristics of subscribers in the intervention arm of randomised controlled trial (RCT) receiving at least 50% of cumulative Kilkari content in the intervention arm of RCT. OBC, other backward classes; SC, scheduled caste; ST, scheduled tribe.
Who is affected by Kilkari?
Programmes may affect healthy behaviours, but do so inequitably; Evidence from other health (non-digital) interventions in maternal, newborn and child health indicate that the greatest benefits have accrued to the population at the top of the socioeconomic scale while the marginalised gain fewer benefits.34 By design, ‘direct to beneficiary’ mobile health communication programmes can only achieve impact among those with access to a mobile phone. This means an analysis of who is left behind by such interventions and how to reach those groups through alternative communication channels and platforms should be undertaken at the intervention design phase to set reasonable expectations of impact.
In addition, considerable heterogeneity exists in the way different sub-groups enrolled in a digital ‘direct to beneficiary’ health communication programme interact with the intervention and respond to it. This necessitates an analysis of the way different subgroups interact and respond to the intervention among those who are eligible and subscribed to it.
Figure 4 presents the Slope of Index Inequality (SII) and Relative Concentration Index (RCI) for different health behaviours between those exposed and not exposed to Kilkari in the intervention arm of the RCT. The RCI depicts the relative disparity in the levels for a range of RMNCH outcomes (health indicators) while the SII depicts the absolute disparity. The magnitude of both relative and absolute summary measures is related to the overall prevalence of the health behaviours. Relative measures like RCI, tend to be larger at lower prevalence levels while absolute measures, like the SII, tend to be low at both very low and very high prevalence levels.35In figure 4, the vertical line represents 0 (no inequity); if the values are positive (to the right of the vertical line) then the health outcome is concentrated in the ‘better off’ and if the values are negative (to the left of the vertical line) then the health outcome is concentrated in the ‘worse off’.
Summary measures of equity for different health behaviours between subscribers who were exposed and not exposed to Kilkari in the intervention arm of the RCT. RCT, randomised controlled trial.
There are three key takeaways from the analysis of the summary measures of equity:
Not all health behaviours are similar in their equity distribution
Findings from the comparison of summary measures suggest that inequity exists for most health indicators among respondents in the intervention arm of the Kilkari RCT (figure 4). Some health indicators like exclusive breastfeeding show very little difference by socioeconomic position while others like sterilisation are concentrated among the worse off. Contraceptive use (modern methods or reversible) and dietary diversity are the most inequitable—in that they are concentrated among the better off. Such variability in the distribution could be ascribed to other factors like cost of adoption (Poor women may have no access to milk substitutes forcing them to rely on exclusive breastfeeding) or other facilitating factors (Sterilisation is promoted primarily by the front-line health workers in the public sector who tend to focus these efforts on the poor)
Kilkari appears to close gaps across wealth quintiles more than across education levels
Inequities exist across both wealth quintiles and educational levels for different health behaviours. In general, the magnitude of the levels of inequity is greater across the wealth quintiles than across education levels—that is, the gap between the poorest and richest is wider than the gap between those with no education and higher education—but this pattern is not consistent (eg, immunisation and breastfeeding indicators). Among those exposed to Kilkari when compared with those not exposed, the inequities in wealth (figure 4 - top panels) linked to many health behaviours are lesser than the inequities in education (figure 4 - bottom panels). This is evidenced by the proximity of the pink dots (level of equity among those exposed to Kilkari to the vertical line than the blue dots (level of equity among those not exposed to Kilkari) in the bottom panels compared with the top panels in figure 4. This means that listening to Kilkari moved the needle much more for the population that was ‘worse off’ in terms of wealth but, simultaneously, ‘better off’ in terms of education (poor but at least started secondary education).
Kilkari appears to allow the ‘worse off’ to make proportional gains in line with the ‘better off’
Among those exposed and not exposed to Kilkari, relative differences (RCI) appear to be smaller than absolute differences (SII)—the blue and pink dots are closer to each other on the left panels than the ones on the right in figure 4. This means that differences in the levels of the health behaviours across wealth quintiles and educational levels between the exposed and not exposed to Kilkari groups were proportional to their actual levels—Kilkari exposure helped ‘lift the boat’ equally (relative to each other) but those ‘better off’ made greater absolute gains due to their pre-existing advantage (figure 4).
The above points are illustrated in figure 5 using the ‘current modern contraceptive use’ indicator as an example, and the two extreme categories in wealth (richest vs poorest) and education (no school vs higher education). The absolute gap in use of modern contraception between those with no schooling and higher education among the exposed and not exposed to Kilkari groups is 30 and 15 percentage points, respectively. The gap among the poorest and richest quintile among the exposed and not exposed to Kilkari is 21 and 37 percentage points, respectively. The relative disparity between the same categories in education for exposed and not exposed is 1.9 and 1.4, while it is 1.5 and 2.6 for wealth. Exposure to Kilkari increases the absolute differential across education levels by 15 points while decreasing by 16 points for wealth quintiles in absolute terms. In relative terms, this translates to a higher differential across education by a magnitude of 0.5 while that for wealth is lower by a magnitude of 1.1. So, exposure to Kilkari appears to benefit the higher educated among those poorest in terms of wealth quintiles.
Prevalence of current modern contraceptive use among Kilkari subscribers in the intervention arm of RCT across education levels and wealth quintiles. RCT, randomised controlled trial.
The bottom line is that health behaviours show wide variation in the distribution across education and wealth. Digital ‘direct to beneficiary’ communication programmes that seek to improve health through behavioural change communication with the population may have differential impact on those who can understand and act on the information provided. Hence the presence of some level of education (from start of secondary education), by priming the population for receipt of information, appears to act as an effect modifier on the pathway to programme impact. The way forward for digital ‘direct to beneficiary’ communication programmes may be to customise and target health information based on the characteristics of specific segments of the population to help close certain gaps in health behaviours.36 This may not be straightforward since it entails collection of subscriber characteristics difficult to measure (wealth) or considered sensitive (ethnicity and social groupings).
Conclusions
By necessity, ‘direct to beneficiary’ mobile health communication programmes target those with phone access and will thus exclude the poorest and most marginalised. Even among those with access to phones, there is inequity among those who can be reached by mobile health interventions, while changes in health behaviours will vary depending on the subgroups exposed, and the type of health behaviour. In the case of Kilkari, women who did not have access to phones or were not registered in government tracking databases were not subscribed to the intervention. Unless the gender gap in mobile phone access is addressed, inequities in the population coverage and reach of Kilkari will persist. Among those enrolled in Kilkari, the programme’s call retry algorithm bolstered efforts to reach the most marginalised but, overall cumulative exposure to content was still lower among the poorest, least educated and the most disadvantaged castes. Research indicates that exposure to Kilkari has helped improve levels of some but not all health behaviours across marginalised groups.25 Education appears to be a key enabler to help improve health behaviours among those exposed to Kilkari content. The study findings also highlight the need for programmes to understand their differential effects on equity through rigorous evaluative methodologies.
Data availability statement
The RCT data are available upon request from Diwakar Mohan (corresponding author) and Amnesty LeFevre (the study PI). NFHS-4 data is available for download from the Demographic and Health Surveys website (https://dhsprogram.com/data/available-datasets.cfm).
Ethics statements
Ethics approval
The Institutional Review Boards of Johns Hopkins Bloomberg School of Public Health, and Sigma in Delhi, India provided the ethical certification for the trial.
Acknowledgments
We thank the women and families of Madhya Pradesh who generously gave of their time to support this work. We are humbled by the opportunity to convey their perspectives and experience. This work was made possible by the Bill and Melinda Gates Foundation. We thank Diva Dhar, Suhel Bidani, Rahul Mullick, Dr. Suneeta Krishnan, Dr. Neeta Goel and Dr. Priya Nanda for believing in us and giving us this opportunity. We also extend our gratitude to BBC Media Action, Beehyv and IMI Mobile without whose tireless efforts the evaluation of the Kilkari programme would not be possible. We additionally wish to thank the larger team of enumerators from OPM-India who worked tirelessly over many months to the implement surveys which form the backbone of our analyses. We additionally thank Vinit Pattnaik at OPM and Erica Crawford at Johns Hopkins for their support to the financial management of our work. Lastly, our figures have been beautified by the great and ever patient Dan Harder of the Creativity Club UK. We thank him for his efforts. **Kilkari Impact Evaluation Team: (listed in Alphabetical order) Smisha Agarwal, Salil Arora, Jean JH Bashingwa, Aarushi Bhatnagar, Sara Chamberlain, Rakesh Chandra, Arpita Chakraborty, Neha Dumke, Priyanka Dutt, Anna Godfrey, Suresh Gopalakrishnan, Nayan Kumar, Simone Honikman, Alain Labrique, Amnesty LeFevre, Jai Mendiratta, Molly Miller, Radharani Mitra, Diwakar Mohan, Deshen Moodley, Nicola Mulder, Angela Ng, Dilip Parida, Nehru Penugonda, Sai Rahul, Shiv Rajput, Neha Shah, Kerry Scott, Aashaka Shinde, Aaditya Singh, Nicki Tiffin, Osama Ummer, Rajani Ved, Falyn Weiss, Sonia Whitehead
References
Footnotes
Handling editor Seye Abimbola
Twitter @jbashingwa, @priydee
Contributors DM: conceptualised the study, wrote the first draft of the manuscript, and analysed the data. KS, SC, AG, PD, OU and AC: provided feedback and helped to write sections of the paper. JJHB and NS: supported the data analysis and provided feedback on the manuscript drafts. AEL: PI for study, conceptualised the study, inputted on the analysis, and helped to write the manuscript.
Funding This study was funded by Bill and Melinda Gates Foundation.
Competing interests All authors have completed the Unified Competing Interest form (available on request from the corresponding author) and declare that the research reported was funded by the Bill and Melinda Gates Foundation. AG, SC and PD are employed by BBC Media Action; one of the entities supporting program implementation. The authors do not have other relationships and are not engaged in activities that could appear to have influenced the submitted work.
Provenance and peer review Not commissioned; externally peer reviewed.