Article Text
PDF
Abstract
Introduction Short message service (SMS) reminders coupled with a small monetary incentive conditioned on prompt vaccination have been shown to improve first-dose measles-containing vaccine (MCV1) uptake. We assessed whether SMS reminders and unconditional monetary incentives—more amenable to programmatic implementation—can improve MCV1 uptake in Kenya.
Methods Caregivers of eligible infants aged 6–8 months were enrolled into an individually randomised controlled trial and assigned to receive either: no intervention (control), two SMS reminders (SMS) sent 3 days, and 1 day before the scheduled MCV1 date, or SMS reminders coupled with a Kenya Shilling (KES) 150 incentive (SMS +150 KES) sent 3 days before the scheduled MCV1 date. Study staff conducted a household follow-up visit at age 12 months to ascertain vaccination status. Log-binomial regression was used to estimate the relative and absolute difference in MCV1 timely coverage (by age 10 months), the primary outcome.
Results Between 6 December 2016 and 31 March 2017, 179 infants were enrolled into each of the three study arms. Follow-up visits were completed between 19 April 2017 and 8 October 2017 for control (n=170), SMS (n=157) and SMS + 150 KES (n=158) children. MCV1 timely coverage was 68% among control arm infants compared with 78% in each intervention arm. This represented a non-statistically significant increase in the SMS arm (adjusted relative risk 1.13; 95% CI 0.99 to 1.30; p=0.070; adjusted risk difference 9.2%; 95% CI: −0.6 to 19.0%; p=0.066), but a statistically significant increase in the SMS + 150 KES arm (1.16; 95% CI 1.01 to 1.32; p=0.035; 10.6%; 95% CI 0.8 to 20.3%; p=0.034).
Conclusion These findings suggest that the effect of SMS reminders coupled with a small unconditional monetary incentive on MCV1 uptake is comparable to that of SMS reminders alone, limiting their utility. Further studies in the absence of unexpected supply-side constraints are needed.
Trial registration number NCT02904642
- measles
- vaccines
- immunisation
This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.
Statistics from Altmetric.com
Summary box
What is already known?
In low-income and middle-income country (LMIC) settings, short message service (SMS; text message) reminders alone can significantly improve timeliness of first dose measles-containing vaccine (MCV1) receipt—that is, vaccination within a short window of the recommended age—but evidence of their impact on the overall proportion of infants receiving MCV1 is mixed.
The effect of SMS reminders coupled with a small conditional monetary incentive on MCV1 timeliness and MCV1 coverage by age 12 months appears to be superior to that of SMS reminders alone.
Unconditional incentives may be more feasibly delivered compared with conditional incentives but their effectiveness under real-world conditions within LMIC immunisation programmes is unknown.
What are the new findings?
SMS reminders coupled with a small unconditional monetary incentive improved MCV1 timeliness but not MCV1 coverage by age 12 months in rural Kenya.
The magnitude of the effect of SMS reminders coupled with an unconditional incentive on MCV1 timeliness and MCV1 coverage by age 12 months was similar to that of SMS reminders alone.
What do the new findings imply?
These findings suggest no added effect of small unconditional monetary incentives over that of SMS reminders alone. However, the findings may have been limited by supply-side barriers to vaccination and additional studies are needed to validate the findings.
This study adds to the body of evidence on the combined effect of SMS reminders and incentives; it is the second to evaluate the effect SMS reminders coupled with any kind of monetary incentive and the first to assess the effect of SMS reminders coupled with unconditional incentives.
Introduction
Measles vaccination is estimated to have prevented 1.3 million global deaths annually between 2000 and 2018 and to have decreased annual measles incidence by approximately two-thirds in the same period.1 However, the full potential of measles-containing vaccine (MCV) to prevent morbidity and mortality remains unrealised. Failure to achieve measles control targets, including elimination, is largely attributable to suboptimal measles vaccination uptake.
In 2019, global coverage for the first and second doses of MCV was 85% and 71%, respectively,2 falling below the threshold of 95% coverage with two MCV doses needed for measles elimination.3 Inequalities in vaccination coverage at the global level—for example, first-dose MCV (MCV1) coverage in 2019 was 69% in WHO Africa Region4 compared with >90% in the WHO Europe, South East Asia and Western Pacific Regions5–7—and at the subnational level are widely acknowledged.8 9
Given high levels of mobile phone ownership and access in low-income and middle-income countres (LMICs)—in 2017 there were 99 and 77 mobile phone subscribers per 100 inhabitants in developing and African countries, respectively10—mobile phone-based interventions may improve demand for routine MCV. Indeed, short message service (SMS; text message) reminders alone or coupled with other interventions have been shown to: increase uptake of diphtheria, tetanus and pertussis (DTP) containing vaccine11–19; increase uptake of MCV17 19 20; improve full immunisation coverage (FIC)12 17–19; and improve vaccine-seeking in general21 among caregivers of infants in a variety of LMICs. In addition, SMS reminders coupled with mobile phone-based incentives (airtime and mobile money (mMoney)) have been shown to improve uptake of DTP-containing vaccine, uptake of MCV, FIC17 and age-appropriate vaccination coverage22 among children in LMICs.
Previously, we showed that SMS reminders coupled with small, conditional mMoney incentives significantly improved measles vaccination timeliness (ie, the proportion of children vaccinated within 2 weeks of the vaccine due date) and measles vaccination coverage (ie, the proportion of children receiving measles vaccination by age 12 months) in a cluster randomised controlled trial conducted in Kenya. In that study, incentives were conditioned on receipt of measles vaccination within 2 weeks of the recommended age.17 Scaling up SMS reminders coupled with conditional incentives in Kenya may not be feasible as it would require real-time monitoring of vaccination receipt in order to determine if caregivers meet the conditions for receiving the incentive. In turn, real-time monitoring of vaccination receipt under real-world conditions is challenged by limitations in the human and/or financial resources needed to implement it. Thus, unconditional incentives which do not require monitoring of vaccination receipt prior to delivery, may be more feasibly delivered at scale to improve measles vaccination coverage.
We sought to evaluate the impact of SMS reminders coupled with an unconditional mMoney incentive on MCV1 timeliness and coverage in Kenya.
Methods
Study design and participants
The Mobile and Scalable Innovations for Measles Immunisation (M-SIMI) study was a three-arm parallel individually randomised controlled trial conducted in Gem subcounty, Siaya County, Kenya. Gem subcounty is a predominantly rural setting with a population of approximately 164 000 in 201623 24 characterised by relatively high malaria, HIV and tuberculosis prevalence and high infant mortality.25 Vaccination coverage was over 90% for third dose DTP-containing vaccine (DTP3) and 84% for MCV1 by age 12 months in 2014–2015.17 The study was conducted in areas within the Kenya Medical Research Institute and Centers for Disease Control and Prevention collaboration’s Health and Demographic Surveillance System (HDSS).
Eligible infants were required to: be aged 6–8 months; be residents of the subcounty as reported by the caregiver and to not have received a dose of routine measles vaccine as indicated in the home-based vaccination record (maternal and child health booklet). Infants’ caregivers were required to not have plans to move within 6 months of enrolment. Mobile phone ownership by the caregiver was not a requirement for enrolment. Participants were randomised and evenly allocated to one of three study arms: (1) control, (2) SMS reminders (SMS), (3) SMS reminders plus a 150 Kenya Shillings incentive (KES; SMS+150; KES150=US$1.50 as of December 2016).
The conduct, analysis and reporting of results were conducted in accordance with the Consolidated Standards of Reporting Trials guidelines.26A detailed description of the methods and protocol has been reported.27
Randomisation and masking
Simple randomisation with an allocation ratio of 1:1:1 to the control, SMS or SMS +150 arm was performed using a list of computer-generated random numbers. Randomisation and preparation of the allocation envelopes were done by the data manager who had no contact with participants. Given the nature of the interventions, study field staff and participants were not blinded to the study arm. The data analyst had access to participants’ study arm allocation during analysis. Additional details on the allocation procedure are provided in the supplement.
Procedures
Community health volunteers (CHVs)—a component of Kenya’s national Community Health Strategy—identified households with children aged 6–8 months and relayed this information to study-employed Community Interviewers (COMM-Is). COMM-Is then visited households to provide general information about the study and to perform screening procedures. COMM-Is verified age eligibility using the date of birth recorded in the home-based vaccination record. Written informed consent was obtained for eligible caregivers. Immediately after enrolment, COMM-Is collected vaccination status, sociodemographic, economic, mobile phone access, mobile phone usage, healthcare utilisation and other general health information from caregivers. Caregivers who did not own a phone were asked to confirm a phone number to which SMS reminders and the mMoney incentive (as applicable) could be sent. Caregivers who could not identify a shared phone number for the study were offered the option to use the COMM-I’s work phone number. All participants received an enrolment SMS, which included a health-related motivational phrase.27
Control arm participants received no interventions. SMS arm participants were sent two SMS reminders; one 3 days before the scheduled MCV1 date (3-day reminder) and the other 1 day before the scheduled measles vaccination date (1-day reminder). SMS+ 150 KES arm participants were sent reminders on the same schedule as the SMS arm participants and were sent the KES 150 incentive 3 days before the scheduled measles vaccination date that is, on the same day as the 3-day reminder.
SMS reminders were sent in the caregiver’s preferred language that is, Dholuo, Kiswahili or English, as indicated at enrolment. The 3-day reminder was comprised of a standard reminder portion, a phrase intended to motivate caregivers and, for SMS+150 KES arm participants, language explaining that the study was sending the incentive to assist with travel expenses. The 1-day reminder was the same across intervention arm participants and consisted of a reminder portion as well as a motivational phrase that was different from the 3-day reminder motivational phrase (online supplemental box S1).27 Caregivers in the control arm were expected to receive one text message (enrolment message only) total and those in the intervention arms were expected to receive three text messages total.
Supplemental material
SMS reminders were sent out automatically using RapidSMS, an open-source platform.28 At enrolment, COMM-Is submitted an SMS to the RapidSMS server containing the infant’s name, infant’s date of birth, and caregiver’s preferred phone number. Based on the information submitted, the 3-day and 1-day SMS reminders, tailored to the applicable study arm and including the infants’ name, were sent from the RapidSMS server to the phone number provided by the caregiver. The Kenya Expanded Programme on Immunisation recommends MCV1 administration at age 9 months.29 Thus, the scheduled measles vaccination date was 274 days (30.42 days/month) from the infant’s date of birth, if falling on a weekday. If falling on a Saturday or Sunday, the scheduled measles vaccination date was defined as the following Monday. RapidSMS was programmed to send reminder messages to a study phone to allow monitoring of per-protocol transmission of SMS. Logs of sent SMS were generated from RapidSMS.
RapidSMS was also programmed to automatically create a cumulative incentive payment list for infants enrolled in the SMS + 150 KES arm. The payment list included infants’ Study IDs, caregivers’ preferred phone numbers and payment dates (ie, 3 days before the scheduled vaccination date). Using the RapidSMS-generated payment list, study staff manually transmitted the KES 150 incentive from a smart phone using the M-PESA mobile money platform operated by Safaricom, one of Kenya’s mobile network providers.
COMM-Is administered a follow-up survey when infants were aged 12 months to collect vaccination status as well as information on caregivers’ opinion of the interventions, reasons for delayed measles vaccination (ie, not vaccinated by age 10 months), incentive use and other general health information. If the child’s vaccinations were not up to date at the follow-up visit, the COMM-I referred the caregiver to the nearest health facility for vaccinations.
Outcomes
The primary outcome was the proportion of infants receiving MCV1 by age 10 months (304 days; ie, MCV1 timely coverage). Secondary outcomes were the proportion of infants receiving MCV1 by age 12 months (365 days; that is, MCV1 coverage) and time to measles vaccination by age 12 months. At the follow-up visit, vaccination status was ascertained from either the home-based vaccination record or the caregiver’s verbal report if a home-based vaccination record was not available. If the home-based vaccination record was available, the date (day, month and year) of vaccination was transcribed and used to calculate the infant’s age at vaccination. For verbal vaccination reports, caregivers specified the month and year of vaccination.
Statistical analysis
The study aimed to measure a≥15 percentage point absolute increase in MCV1 timely coverage in the intervention arms compared with the control arm. We presumed that a≥15% increase in the proportion of children receiving MCV1 by age 10 months would represent a meaningful effect from a policy-maker perspective. Based on previous coverage estimates in the study area,17 we assumed MCV1 timely coverage of 70% in the M-SIMI control group. We also assumed a type 1 error (alpha) of 0.05, a power (1-beta) of 0.80, yielding a sample size of 134 infants per study arm after application of a continuity correction. The sample size was adjusted to account for up to 25% lost to follow-up, which included death, outmigration and verbal report of measles vaccination at 10 months of age. A priori, verbal immunisation reports at the follow-up visit were to be excluded from the analytic sample. After accounting for potential losses to follow-up, the estimated sample size to assess the primary outcome was 537 infants total, or 179 infants enrolled per study arm.
Log-binomial regression was used to estimate the relative risk (RR), difference in risk (RD) and respective 95% CI of measles vaccination by age 10 months or age 12 months in each of the intervention arms compared with control arm. For assessment of the primary endpoint (MCV1 timely vaccination), children were censored at age 10 months. Any predictive baseline characteristics determined to be unequally distributed at the 5% significance level were included as covariates in the regression model to adjust for potential confounding. The primary endpoint was analysed according to intention-to-treat (ITT) principles. A modified per-protocol sensitivity analysis for the primary endpoint was also conducted. The modified per-protocol analysis was defined as dispatch of two SMS reminders as per the target schedule that is, 3 days and 1 day before the scheduled vaccination date.
In order to evaluate whether baseline participant characteristics modified the effect of the interventions on the likelihood of timely measles vaccination, we performed stratified (subgroup) analysis. To identify independent variables to include in the subgroup analysis, a risk factor analysis of baseline participant characteristics associated with MCV1 vaccination by age 10 months was conducted among only control arm participants using univariate log-binomial regression. Characteristics significant at the 10% level were included in subgroup analysis. Stratification by mobile phone ownership status and travel time to the health facility were pre-specified in the study protocol. Stratification by maternal education was added post hoc, based on evidence of an association between higher educational attainment and lower likelihood of missed measles vaccination in the study area.17 The significance level for subgroup analysis was 5%.
Survival analysis was performed to assess whether time to measles vaccination differed significantly across the study arms. Time origin was defined as enrolment and events were right censored at age 365 days. The cumulative probability of measles vaccination was plotted using the failure functions estimated using the Kaplan-Meier method. Equality of the cumulative incidence functions were tested using the log-rank test.
Analyses were performed using Stata/SE, V.14.1 (StataCorp).
Patient and public involvement
This study did not involve patients. We did not directly include participants or public representatives in the design of this study, but the design was informed by focus group discussions among community members for a similar, related study previously conducted in the same area.30 No participants or public representatives contributed to selection of outcome measures. CHVs, who are community members, were involved in identification of study participants. Study findings were presented to CHVs.
Results
CHVs identified 639 potentially eligible infants between 6 December 2016 and 31 March 2017. Of those, 537 infants—the target sample size—were randomised to the control, SMS and SMS + 150 KES arms (179 infants each). Follow-up visits were completed for control (n=170), SMS (n=157) and SMS + 150 KES (n=158) infants between 19 April 2017 and 8 October 2017. The analytical sample included 160 (89%), 146 (82%) and 149 (83%) of all infants enrolled in the Control, SMS and SMS + 150 KES arms, respectively. Reasons for exclusion from enrolment, loss to follow-up and exclusion from the analytical sample are provided in figure 1. The 82 participants who were excluded from the analytical sample were similar to participants in the analytic sample except for birth order and maternal age (online supplemental table S1). Maternal age was included as an independent variable in regression models because it was unevenly distributed across study arms (table 1) and has previously been shown to be a determinant of childhood vaccination status within the study area.31 The per-protocol analysis included 126 SMS and 126 SMS + 150 KES infants, representing 86% and 85% of the analytical sample, respectively.
Screening, enrolment and follow-up flow diagram for the M-SIMI study. M-SIMI, Mobile and Scalable Innovations for Measles Immunisation; SMS, short message service; MCH card, maternal and child health card (home-based vaccination record).
Baseline characteristics of infants in the analytical sample, Gem subcounty, Kenya (2016–2017)
MCV1 timely coverage was 68% (109 of 160) among Control arm infants compared with 78% (114 of 146 in SMS only; 116 of 149 in SMS + 150 KES) in both the intervention arms. This represented a non-statistically significant increase in the SMS arm (adjusted RR (aRR) 1.13; 95% CI 0.99 to 1.30; p=0.070; adjusted risk difference (aRD) 9.2%; 95% CI −0.6 to 19.0%; p=0.066), but a statistically significant increase in the SMS + 150 KES arm (aRR 1.16; 95% CI 1.01 to 1.32; p=0.035; aRD 10.6%; 95% CI 0.8 to 20.3%; p=0.034; table 2). In the per-protocol analysis, MCV1 timely coverage in the intervention arms was similar to ITT MCV1 timely coverage and findings were comparable to the ITT analysis with the exception that the absolute increase in MCV1 timely coverage among SMS infants achieved statistical significance (table 2). None of the characteristics identified in the analysis of predictors of vaccination in the Control arm (phone ownership, age at enrolment, birth order and time to health facility; online supplemental table S2) modified the effect of the interventions on MCV1 timely coverage. Post hoc, maternal age was added to the subgroup analysis and similar to the other variables assessed, was not found to modify the effect of the interventions (figure 2). MCV1 coverage was not significantly higher in the SMS (84%; 125 of 160) or SMS + 150 (85%; 123 of 146) arms compared with the Control arm (78%; 126 of 149; table 2).
Subgroup analysis of the impact of SMS reminders with or without KES 150 incentiveon MCV1 timely coverage, by study arm. Relative risks (RR) adjusted for maternal age* Interaction term p value. KES, Kenya Shilling; MCV, measles-containing vaccine; SMS, short message service.
Measles vaccination timeliness by age 10 and coverage by age 12 months in intervention arms compared with the control arm
The median time to measles vaccination was age 286 days (IQR 276–324 days) in the control arm, age 284 days (IQR 276–298 days) in the SMS arm and age 282 days (IQR 275–302 days) in the SMS + 150 KES arm (figure 3). There was no significant difference in time to measles vaccination across study arms (log-rank test p=0.182; online supplemental figure S1) even after performing testing stratified by maternal age (maternal age ≤25 years log-rank test p=0.195, maternal age >25 years log-rank test p=0.576, stratified log-rank test p=0.158; data not shown).
Cumulative incidence of measles vaccination by age 12 with time origin as age at enrolment. MCV, measles-containing vaccine; SMS, short message service.
The reason for delayed vaccination was obtained for 82% (95 of 116) of infants with delayed MCV1. Caregivers who were not queried were missed because the interviewer did not identify them as having delayed vaccination. Of the infants whose caregivers were queried, common (>10%) reasons for not receiving MCV1 by age 10 months were an ongoing nurses strike (36%) and vaccine stock-out (16%). Other reported reasons for delayed vaccination are provided in table 3. There was no significant difference in cause of delayed vaccination across study arms (overall χ2 p=0.529).
Reasons for delayed measles vaccination
Reasons why reminders were not sent per-protocol are shown in online supplemental table S3. Whereas at least one SMS reminder was sent out by the study team to participants, 67% (98 of 146) and 81% (120 of 149) of SMS arm and SMS + 150 KES arm caregivers, respectively, reported that they received at least one SMS reminder. The proportion of caregivers who did not know the total number of reminders received was significantly higher among those sharing a phone compared with those who owned the phone receiving messages (table 4). Due to an error in the questionnaire skip pattern, caregivers who reported not receiving any reminders were not queried as to whose phone the reminders were sent. More than 85% of caregivers in each of the intervention arms felt that SMS reminders influenced the decision to vaccinate (table 4). mMoney incentives were sent to all participants in the SMS +150 KES arm. Ninety-one (61%) incentives were sent out 3 days before the scheduled vaccination date, as intended in the study protocol, and 127 (85%) before the scheduled vaccination date (online supplemental table S4). Of 149 SMS + 150 KES arm caregivers, 71% (n=105) reported that they received the mMoney incentive, of whom 72% (76 of 105) owned the phone to which the mMoney incentive was sent. Receipt of the incentive was reported to have influenced the decision to vaccinate in 84% (88 of 105) of these caregivers. A little over three-quarters of caregivers cashed out the incentive within 3 days of receiving the incentive. Among infants whose caregivers who cashed out the incentive, 19% (20 of 103) and 13% (13 of 103) did not receive MCV1 by age 10 months and age 12 months, respectively (online supplemental table S5). Approximately 95% of caregivers reported a positive experience related to receiving the incentive. Only one caregiver reported that they would be less likely to seek vaccination in the absence of an incentive. Incentives were commonly used to cover transport costs (n=59; 56%), housing expenses (n=21; 20%) and to purchase food (n=16; 15%; table 4).
Reported receipt of SMS reminders, opinions about SMS reminders, experience with and opinions about incentives
Discussion
When delivered under real-world conditions, SMS reminders coupled with a small unconditional monetary incentive increased the timeliness of MCV1 receipt, as did SMS reminders despite not achieving statistical significance. However, MCV1 timely coverage was significantly higher in the SMS arm compared with the control arm in the per-protocol analysis, suggesting that when delivered under ideal conditions, SMS reminders alone may improve MCV1 timeliness. Though the effect of the interventions on MCV1 timely coverage was modest—the findings suggested the potential to improve timeliness by 9%–11% points—the magnitude of effect could be meaningful from a policy perspective as delay in vaccination increases the proportion of susceptible persons, of particular clinical concern in high HIV prevalence settings32 such as this one. There was no evidence that the interventions exacerbated or reduced inequities in MCV1 timeliness. The effects of the interventions on MCV1 coverage by age 12 months were even more modest and did not achieve statistical significance, leaving unclear whether either intervention may reduce the proportion of infants not receiving routine MCV.
This study was novel in that it evaluated the combined impact of SMS reminders and unconditional monetary incentives on vaccination timeliness and coverage. To our knowledge, it is the first study to evaluate the combined effect of those interventions. It is only the second study, following our previous evaluation that provided conditional monetary incentives,17 to evaluate the impact of SMS reminders coupled with any type of monetary incentive on vaccination uptake; other studies have assessed the impact of SMS reminders11–16 18 20 33 34 and incentives separately.35–37 Furthermore, we illustrated that CHVs can support community-based identification of infants targeted for immunisation interventions.
The finding from this study that SMS reminders coupled with an incentive significantly increase measles vaccination timeliness is consistent with the M-SIMU study17 and reproduces a positive finding within the same study population. However, the findings also suggest differential impact of unconditional vs conditional incentives on MCV1 uptake. First, the magnitude of the impact of SMS reminders coupled with an unconditional monetary incentive on MCV1 timely coverage in this study—an approximately 9%–11% absolute increase—was less than the 21% absolute increase associated with SMS reminders coupled with conditional incentives in the M-SIMU study. Second, the finding that the effect of SMS reminders coupled with an unconditional incentive was similar to that of SMS reminders alone was in contrast to findings from the M-SIMU study where the effect of SMS reminders coupled with conditional incentives surpassed that of SMS reminders alone.17
There is mixed evidence on whether the impact of conditional incentives on health outcomes varies from that of unconditional incentives. Systematic reviews have found positive effects of both conditional and unconditional financial incentives for outcomes such as HIV prevention and treatment and healthcare seeking.38–40 Some studies have found that unconditional incentives improve return or response rates for health-related surveys whereas conditional incentives do not, though not all incentives were financial.41–43 In terms of vaccination outcomes, evaluations of large conditional and unconditional cash transfer programmes in Central and South America found that transfers significantly increased uptake for some childhood vaccines but not for others, with neither type of cash transfer programme having consistent effects.44–47 Three studies of de facto conditional (incentives were delivered at immunisation visits) non-cash incentives in Pakistan and India found that the incentives significantly increased DTP3 or overall vaccination uptake.22 35 37 In the one cluster randomised controlled trial that simultaneously assessed the impact of both conditional and unconditional cash transfers on childhood health in Zimbabwe, neither intervention significantly improved vaccination uptake.48 Contrary to the suggestion from this study that conditional incentives had a greater effect compared with unconditional incentives, conditional incentives have been theorised to negatively impact intrinsic motivation in the behavioural economics literature.49 Further research is needed to evaluate whether, in a setting where intrinsic motivation to practice the target behaviour is fairly high—in this setting coverage for third-dose DTP-containing vaccine was 96% (table 1)—conditional incentives may in the short run improve outcomes through increasing extrinsic motivation.
At the same time, methodological factors (and not conditionality of the incentive) may explain differences in the effects in this study compared with those observed previously in the M-SIMU study. Whereas SMS reminders plus conditional incentives were associated with a statistically significant 6% absolute increase in MCV1 coverage by age 12 months in the M-SIMU study, the 7% absolute increase in the SMS + 150 KES arm of this study was not statistically significant. However, this study was underpowered to detect the observed increase, as we assumed an absolute increase in MCV1 timely coverage of ≥15% in sample size calculations. To be sufficiently powered to detect the approximately 7% absolute increase in MCV1 coverage by age 12 months observed, a sample size of approximately 593 individuals in each study arm would have been required. Second, the incentive amount and nature of the incentive differed in this study compared with the M-SIMU study. The incentive amount in this study was KES 150 (~US$1.87 in August 2015; indexed to 2015 KES) compared with KES 200 (~US$2.35 in August 2015; indexed to 2015 KES) in the M-SIMU study. Furthermore, in this study, caregivers received a maximum monetary incentive of KES 150 compared with as much as KES 800 in the M-SIMU study as that study incentivised the pentavalent vaccine primary series in addition to MCV1.17 The higher incentive amount in the M-SIMU study may have been more motivating to caregivers given that the value of incentives is thought to be positively correlated to their effect.49 Furthermore, formative research for the M-SIMU study found that 93% of caregivers felt that KES 200 would motivate prioritising attending a vaccination visit compared with 83% for KES 150 (D. Gibson personal communication). Finally, the repeated vaccination prompts during the M-SIMU study (SMS reminders for the pentavalent series and MCV1) may have induced greater vaccine seeking among those caregivers as more frequent reminders may be associated with greater impact.50
Beyond research design, a nationwide nurses’ strike that began on 5 June 201751 and was ongoing at the time of study completion, represented an unexpected supply-side barrier to vaccination which may have affected the study findings. Approximately three-quarters and one-third of the analytic sample reached 10 months of age and 12 months of age, respectively, before the strike began. We observed that the 78% MCV1 coverage estimate in the Control arm was lower than the expected 83%–84% coverage based on previous estimates of coverage in the study area17 31 and in Siaya County.52 Exclusion of infants that had received a dose of MCV prior to reaching age 9 months would not explain the discrepancy; had they been enrolled and followed up (a conservative assumption), Control MCV1 coverage would have been ~80%. Although MCV1 coverage was not significantly higher in intervention infants who reached age 12 months before the strike compared with control infants who also reached age 12 months before the strike (online supplemental table S6), in theory, the nurses’ strike may have reduced the potential impact of interventions if intervention arm caregivers seeking MCV1 in public health facilities were unable to get their infants vaccinated as a result of the strike. Of note, estimates of MCV1 timely coverage and coverage by age 12 months in the intervention arms in the period before the nurses’ strike were comparable (online supplemental table S6).
This study has several limitations. First, the analytic sample comprised 85% of enrolled participants. Excluded infants were more likely to be firstborn children and to have mothers aged 25 years or less. Similar patterns were observed previously and are attributed to cultural practices around pregnancy and birth whereby mothers return to their rural home to receive support caring for newly-born infants.17 Firstborn infants and children of younger mothers are thought to be less at risk of being unvaccinated or receiving vaccination with delay than later-born infants and children of older mothers, respectively.31 53–56 Thus, the effect estimates may have been biased towards the null. Another limitation of this study is that only 85% of SMS reminders and 61% of mMoney incentives were sent out as intended which may have biased the effect toward the null. Indeed, the per-protocol analysis but not the ITT analysis found SMS reminders significantly improved MCV1 timeliness. Delivery of SMS reminders was hampered mainly by power outages; therefore, the findings may provide some insight into the real-world effectiveness of SMS reminders when delivered within a programmatic setting likely to experience similar challenges. mMoney incentives were delayed because they were sent out manually by study staff during weekdays but not on weekends when staff were not working. In post hoc analyses, delayed delivery of the incentive did not impact MCV1 timely coverage (online supplemental tables S7 and S8). A related limitation is that≥20% of caregivers may not have received the interventions, based on self-report. If true, the observed effects would be biased towards the null. We did not have objective mechanisms to verify intervention receipt and self-report was susceptible to recall bias as we asked caregivers whether they had received the interventions about 3 months after they had been sent. Besides recall bias, reported receipt of interventions may have been influenced by phone ownership, whereby caregivers who owned a phone may have been more likely to remember having received the intervention compared with those sharing the phone to which interventions were sent. Curiously, the proportion of SMS + 150 KES caregivers who reported receiving SMS reminders was higher than the proportion who reported receiving the incentive. This could perhaps be explained if SMS messages were more likely to be passed along than the incentive for SMS + 150 KES caregivers who shared a phone. An additional limitation of this study is that COMM-Is, who collected measles vaccination status at the follow-up visit, were responsible for assigning the study arm at enrolment and were therefore not blinded to study arm allocation. We think ascertainment bias is unlikely because follow-up occurred 4–6 months from enrolment and we structured the follow-up questionnaire to collect vaccination data prior to any questions that identified the study arm allocation. Repeat follow-up interviews by the study’s field supervisor support this; of the 5% that was reinterviewed, only one vaccination discrepancy was found.
Conclusion
This study’s findings suggest that the impact of SMS reminders coupled with a small unconditional monetary incentive on MCV1 timeliness was comparable to that of SMS reminders alone, yet it would be more costly to implement. In addition, the impact of SMS reminders coupled with small unconditional incentives on reducing the proportion of measles-unvaccinated infants was unclear, though the findings suggest that any effect is likely to be modest and of similar magnitude to that of SMS reminders alone. Notably, the similarity in MCV1 uptake levels among infants in the intervention arms was observed even before the nurses’ strike and other supply-side constraints observed in the study are likely to be experienced under real-world conditions in LMICs. Additional studies in the absence of unexpected supply-side constraints, such as the nurses’ strike, are needed to inform the generalisability of the findings.
Acknowledgments
We thank the study participants, Gem community, KEMRI/CDC-Kisumu, Community Health Volunteers (Gem Sub-County) and project staff. The authors also thank Danet Opot for programming the RapidSMS system. This paper is published with the agreement of the Director of the Kenya Medical Research Institute.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Handling editor Senjuti Saha
Contributors DGG, EWK, BO, JW and KH conceived the study. DGG, BO and EWK developed the study protocol. DGG, EWK and KLO and BO provided scientific and study oversight. All authors generated hypotheses, interpreted the data and critically reviewed the manuscript. JW generated the randomisation sequences. EWK wrote the first draft of the manuscript and conducted statistical analyses. DGG and EWK conducted the literature review. BO supervised field activities, including data collection.
Funding This study was funded by the Bill & Melinda Gates Foundation. The corresponding author had full access to all the data in the study and had final responsibility for the decision to submit for publication.
Disclaimer The funder of the study had no role in study design, data collection, data analysis, data interpretation, or writing of the report.
Competing interests None declared.
Patient consent for publication Not required.
Ethics approval The study was approved by the KEMRI Scientific and Ethics Review Unit (SERU; KEMRI/SERU/CGHR/003/3311). The Johns Hopkins Bloomberg School of Public Health Institutional Review Board (JHSPH IRB) deferred ethical oversight to KEMRI SERU.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Data will be shared contingent on ethical and institutional approvals.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.