You are here

  • Home
  • Archive
  • Volume 6, Issue 7
  • Short message service (SMS) reminders for childhood immunisation in low-income and middle-income countries: a systematic review and meta-analysis

Article Text

Article menu
XML
Original research
Short message service (SMS) reminders for childhood immunisation in low-income and middle-income countries: a systematic review and meta-analysis
  1. Paul Eze1,
  2. Lucky Osaheni Lawani2,
  3. Yubraj Acharya1
  1. 1Department of Health Policy and Administration, Pennsylvania State University, University Park, Pennsylvania, USA
  2. 2Institute of Health Policy, Management & Evaluation, University of Toronto, Toronto, Ontario, Canada
  1. Correspondence to Paul Eze; peze{at}psu.edu

Abstract

Introduction Childhood vaccine delivery services in the low- and middle-income countries (LMIC) are struggling to reach every child with lifesaving vaccines. Short message service (SMS) reminders have demonstrated positive impact on a number of attrition-prone healthcare delivery services. We aimed to evaluate the effectiveness of SMS reminders in improving immunisation coverage and timeliness in LMICs.

Methods PubMed, Embase, Scopus, Cochrane CENTRAL, CINAHL, CNKI, PsycINFO and Web of Science including grey literatures and Google Scholar were systematically searched for randomised controlled trials (RCTs) and non-RCTs that evaluated the effect of SMS reminders on childhood immunisation and timeliness in LMICs. Risk of bias was assessed using the Cochrane Risk of Bias 2.0 assessment tool for RCTs and Cochrane Risk of Bias in Non-randomised Studies of Interventions tool for non-RCTs. Meta-analysis was conducted using random-effects models to generate pooled estimates of risk ratio (RR).

Results 18 studies, 13 RCTs and 5 non-RCTs involving 32 712 infants (17 135 in intervention groups and 15 577 in control groups) from 11 LMICs met inclusion criteria. Pooled estimates showed that SMS reminders significantly improved childhood immunisation coverage (RR=1.16; 95% CI: 1.10 to 1.21; I2=90.4%). Meta-analysis of 12 included studies involving 25 257 infants showed that SMS reminders significantly improved timely receipt of childhood vaccines (RR=1.21; 95% CI: 1.12 to 1.30; I2=87.3%). Subgroup analysis showed that SMS reminders are significantly more effective in raising childhood immunisation coverage in lower middle-income and low-income countries than in upper middle-income countries (p<0.001) and sending more than two SMS reminders significantly improves timely receipt of childhood vaccines than one or two SMS reminders (p=0.040).

Conclusion Current evidence from LMICs, although with significant heterogeneity, suggests that SMS reminders can contribute to achieving high and timely childhood immunisation coverage.

PROSPERO registration number CRD42021225843.

  • systematic review
  • vaccines
  • child health

Data availability statement

Extracted data are available, on request, from the corresponding author.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/bmjgh-2021-005035

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Key questions

    What is already known?

    • Childhood immunisation coverage in several low/middle-income countries (LMICs) remains short of the WHO’s Global Vaccine Action Plan 2020 goal of 90% childhood immunisation coverage.

    • LMICs have witnessed exponential increase in mobile phone usage in the last decade and mobile phone short message service (SMS) reminders have also shown positive impact on attrition-prone healthcare delivery services.

    What are the new findings?

    • Meta-analysis shows that SMS reminders significantly improved childhood immunisation coverage and timeliness in the intervention groups compared with control groups.

    • However, SMS reminders are significantly more effective in improving childhood vaccination coverage in low-income countries and lower middle-income countries than in upper middle-income countries, mainly due to ceiling effect.

    • Sending more than two SMS reminders at least 24 hours before the appointment date ensures timely receipt of childhood vaccines than sending one or two SMS reminders.

    What do the new findings imply?

    • SMS reminders could be effective in improving overall vaccination coverage and timeliness in LMICs. This is especially significant in lower middle-income and low-income countries where the highest numbers of unvaccinated children live, and children are most likely to miss out on lifesaving vaccines.

    Introduction

    Despite significant improvements made in the global childhood immunisation coverage in the last two decades, 14 million children worldwide still missed out on lifesaving vaccines in 2019.1 While global childhood immunisation coverage reached 85%, and 125 countries reached at least 90% childhood immunisation coverage in 2019,1 childhood immunisation coverage in several World Bank-defined low/middle-income countries (LMICs) remained short of the WHO’s Global Vaccine Action Plan 2020 goal of 90% childhood immunisation coverage.2–4 Sixty-three of the 69 WHO member countries that are yet to reach at least 90% DPT-3 (third dose of diphtheria, pertussis and tetanus) vaccine coverage in 2019 are LMICs.1 2 Countries in the World Bank-defined lower middle-income category and low-income category had DPT-3 coverage of 84% and 74%, respectively, in 2019.1 5 With this current trend, none of the LMICs will meet the sustainable development goals (SDG) 2030 childhood vaccination coverage targets for DPT and measles.6 In particular, countries in sub-Sahara Africa (SSA) account for approximately 25% of the annual global births, but SSA children contribute to 45%–50% of severe morbidity and fatalities worldwide from leading vaccine preventable diseases.7 One in five SSA children goes without lifesaving vaccines.8

    SDG 2030 priorities include eliminating vaccine preventable diseases and improving access to new lifesaving vaccines.6 Achieving these goals requires continuously functioning childhood immunisation programmes,9 as shown in online supplemental file 1. Childhood immunisations are often delayed or missed either due to caregivers’ lack of awareness about the vaccines or their due dates.10 Many LMICs do not have functioning primary healthcare systems and routine well childcare services, making a somewhat complicated primary childhood vaccine series (with multiple appointments at various ages) difficult for caregivers to remember.8 9 Besides missing an opportunity to protect the child with lifesaving vaccines and the immense benefits thereafter,11 12 missing scheduled childhood immunisation appointments has financial and human resource implications leading to inefficiency in healthcare delivery.10 Immunisation reminders have been shown to improve compliance and timeliness, but studies suggest that traditional reminders have a low impact.13–16 Reminder systems work through a variety of mechanisms including phone calls, letters, postcards and email meant to prompt the patient.17 Although most types of reminder systems are effective, mobile phone reminders have been found to be most effective.14 18

    Supplemental material

    Recent explosion of mobile phone usage transcends age, gender and state boundaries, and has significant potential for improving health.17–19 Mobile phones have become integral parts of daily life. Wireless technologies cover over 95% of the global population.20 LMICs have witnessed an exponential increase in the number of mobile phone users.21 Among the regions, with over 90% of the population covered by 2G networks at the end of 2017,22 and over 770 million mobile cell subscriptions in 2018,23 SSA is the fastest growing mobile region in the world. Mobile phone penetration (defined as the number of SIM cards or mobile phone numbers per 100 people in a region) in SSA was 82% in 2018.23 India and Pakistan, together, have over a billion mobile phone subscribers: over 870 million and over 130 million, respectively.23 24

    Short message service (SMS) reminders are also important utilities in a new system of healthcare delivery called mobile health (mHealth) which arguably is one of the key factors shaping the future of healthcare.25 26 A number of recent systematic reviews have demonstrated positive impact of SMS reminders on a variety of attrition-prone healthcare delivery, including antenatal care,27 healthcare appointment,10 28 adherence to HIV medications29 and adherence to chronic disease medications.30 Although SMS reminders have also been shown to have a positive impact on vaccination in children, adolescents and adults in high-income countries,31–34 trials in LMICs have not been comprehensively pooled and analysed.

    Given the increasing ubiquitousness of mobile phones in LMICs, the demonstrated utility of SMS reminders in reducing attrition in other healthcare delivery services, the documented success of SMS reminders in improving vaccination coverage in high-income countries and the pressing need for interventions to boost childhood immunisation coverage and timeliness in LMICs, it is important to evaluate interventional studies on SMS reminders in LMICs. Therefore, this study aims to evaluate the effectiveness of SMS reminders in improving childhood immunisation coverage and timeliness in LMICs, thus generating evidence to support governments and development partners in strengthening existing vaccine delivery structures in LMICs.

    Methods

    The protocol for this systematic review was registered on PROSPERO database (CRD42021225843), and the review findings were reported according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.35

    Inclusion criteria

    We included articles published in any language that were peer reviewed and that met the eligibility criteria based on the PICOS strategy (online supplemental file 2—inclusion criteria):

    • Population (P): the study used mothers of children less than 24 months from LMICs, as classified by the World Bank in 2020,36 as the population.

    • Intervention (I): interventions in which SMS/text messages provide reminders related to vaccinations. Interventions were included even if the effectiveness of SMS reminders on childhood immunisation coverage or timeliness was not the primary intervention assessed or was assessed in conjunction with other interventions to improve demand for childhood immunisation (such as conditional cash transfers or other forms of reminder interventions).

    • Comparison (C): the study compared the intervention with usual care in which mothers are reminded verbally at the health centre or the next appointment was written on the child’s health card.

    • Outcome (O): the study evaluated the effectiveness of SMS reminders on DPT-3, Penta-3 or overall childhood immunisation uptake and/or timeliness. We accepted whichever method by which the outcome was assessed in the included intervention trials/studies, including by mothers self-report, home-based health records or facility-based childhood immunisation register.

    • Study design (S): the study was either a randomised controlled trial (RCT) or a non-RCT interventional study.

    Search strategy and study selection

    We searched PubMed, Embase, Scopus, Cochrane CENTRAL, CINAHL, CNKI, PsycINFO and Web of Science for RCTs and non-RCTs of the effects of mobile phone SMS reminders on childhood immunisation published from 1 January 2000 to 30 November 2020 in 135 World Bank-defined LMICs,36 but later updated the search to 31 December 2020. We used search terms covering short message services reminders (SMS, texts, text message), childhood immunisation and LMIC (online supplemental file 3—search strategy). We also searched trial registries (eg, Pan African Clinical Trial Registry, ClinicalTrials.gov and Chinese Clinical Trial Registry) for relevant registered trials; grey literature websites (eg, New York Academy of Medicine Grey Literature and Open Grey); prepublication server deposits (eg, medRxIV and bioRxIV) and Google Scholar. We also sought for relevant articles from the references of studies identified through the database search. There was no language restriction and non-English studies were translated into English using a translation service.

    The search was independently conducted by two authors (PE and LOL) and duplicate articles from different database searches were excluded. The two authors first underwent a moderation exercise to ensure uniform application of inclusion criteria. Then, titles and abstracts were assessed applying the inclusion criteria. Discrepancies were resolved by discussion. Finally, full text of each remaining articles was assessed against the inclusion criteria.

    Data extraction

    Two authors (PE and LOL) extracted information from the included studies including the country in which the study was conducted, study design, intervention details, study participant characteristics and setting, sample size and outcome(s) relative to childhood immunisation coverage. Where studies reported both intention-to-treat and per-protocol analyses, we used the intention-to-treat data. Where studies reported data for both overall immunisation and DPT-3 data, we extracted data for overall immunisation. However, we also included studies that reported only DPT-3 outcomes, given the conventional use of DPT-3 to monitor progress of interventions aimed at improving vaccine delivery services.12 37 DPT-3 coverage—defined as the proportion of children receiving complete (three) doses of diphtheria, pertussis and tetanus—is a particularly valuable measure of the childhood immunisation coverage and countries’ vaccine delivery effectiveness.1 37

    Authors of eligible unpublished concluded trials identified from trial registries were contacted to provide results for inclusion in the study. Studies that employed cluster sampling were reduced to their effective sample size using the reported design effect and intracluster correlation coefficient before entry into statistical software.38 Microsoft Excel was used to organise extracted data from included studies. Disagreements were resolved through discussion until there was 100% agreement.

    Risk of bias assessment

    Two authors (PE and LOL) independently used the Cochrane’s Risk of Bias 2.0 tool for assessing risk of bias39 to assess included RCTs and their respective protocols and trial registry records for risk of bias in five domains: (1) bias arising from the randomisation process, (2) bias due to deviations from intended interventions, (3) bias due to missing outcome data, (4) bias in measurement of the outcome and (5) bias in selection of the reported result. If any of the five domains was found to be associated with some concerns of risk of bias or high risk of bias, the overall risk of bias was rated as ‘some concern’ or ‘high risk’, respectively. Otherwise, RCTs were rated as ‘low risk’. Disagreements were resolved by discussion.

    Likewise, two authors (PE and LOL) independently assessed risk of bias in non-RCTs across seven domains using the Cochrane Risk Of Bias In Non-randomised Studies of Interventions (ROBINS-I) tool.40 Overall risk of bias was rated as ‘low risk’, ‘moderate risk’ or ‘serious/critical risk’. Disagreements were also resolved by discussion. Risk of bias assessment for both randomised and non-randomised studies were then presented graphically using the Risk-Of-Bias VISualization tool.41

    Measures of treatment effect and heterogeneity

    Risk ratio (RR) was used as a measure of treatment effects and reported with 95% CIs. The primary outcome was change in childhood immunisation coverage while the secondary outcome was change in timelessness of childhood immunisation vaccines. Clinical heterogeneity (ie, variability in participants, interventions, outcomes studied) and methodological heterogeneity (ie, variability in study design and risk of bias) of included studies were characterised using descriptive statistics. Statistical heterogeneity (ie, variability in the intervention effects) was reported using the I2 statistic.

    Data synthesis

    We performed data analysis according to the guidelines specified in the Cochrane Handbook for Systematic Reviews of Interventions.38 Pairwise meta-analysis using the conventional random effects (DerSimonian-Laird) model was performed to pool data from individual trials and reported as pooled RR with 95% CI.38 42 Analyses were conducted using Stata V.16.1 (STATA Corp). Predictive interval was calculated using Prediction Intervals programme provided by BIOSTAT (Englewood, New Jersey, USA).43 An α of 0.5 was used as the cut-off for statistical significance.

    Sensitivity analysis was first performed using fixed-effect (Mantel-Haenszel) model which offers ‘the best estimate of the intervention effect’,38 as it attributes more weight to more precise studies.42 Then, sensitivity analysis was also performed using the random effects restricted maximum likelihood (REML) model.44 Robustness of pooled estimate was also assessed for the influence of studies with less than 200 participants in either arm—as studies with small sample size are more likely to exaggerate the intervention’s effect size.38 Pooled estimate was also assessed for influence of studies with sample size outliers. Lastly, sensitivity of pooled estimate was assessed for influence of studies that reported only DPT-3 data.

    Subgroup analyses were performed by the countries’ income status, study setting (urban vs rural), study design, number of SMS reminders sent per scheduled visit, timing of last SMS reminder and study quality. Meta-regression was performed to assess the modifying effect of country’s income status, study setting, study design, number of SMS reminders sent and time last SMS reminders was sent on the intervention effect. Finally, evidence of publication bias was assessed by examining the symmetry of the funnel plot and performing Harbord test.

    Assessment of certainty of evidence

    The certainty of evidence was assessed using Grading of Recommendations, Assessment, Development and Evaluation on the GRADEPro platform.45 For each outcome, the quality of the body of evidence was assessed as high, moderate, low or very low; based on the following criteria: risk of bias, heterogeneity, imprecision, indirectness and publication bias.38 45

    Results

    Selection of studies

    The study selection process is illustrated in a PRISMA flow diagram (figure 1). The database search yielded a total of 5480 studies, and we included seven additional studies after manual searches in Google Scholar, grey literature, clinical trial registries and tracking references of selected primary articles. A total of 759 duplicates were removed and the titles and abstracts of the remaining 4728 studies were screened based on the exclusion criteria. Most studies (n=4689) were excluded. After eligibility and critical appraisal of the full texts of 39 records, 18 studies met the inclusion criteria for data extraction, qualitative synthesis and meta-analysis,46–63 while 21 studies were excluded for the following reasons: intervention was not SMS reminder (n=9),64–72 study setting was not an LMIC (n=2),73 74 conference abstract (n=2),75 76 literature reviews (n=2),77 78 could not extract data on specific effect of SMS reminder (n=2),79 80 observational case–control study (n=1),81 no comparator/control arm (n=1),82 control arm not usual care (n=1)83 and outcome was not DPT-3, Penta-3 or overall immunisation (n=1).84

    Figure 1
    Figure 1

    PRISMA flow diagram showing studies selection process. DPT-3, third dose of diphtheria, pertussis and tetanus; LMICs, low/middle-income countries; PRISMA, Preferred Reporting Items for Systematic Reviews and Meta-Analyses; SMS, short message service.

    Characteristics of included studies

    Table 1 summarises the main characteristics of the 18 studies included in the analysis. Of the 18 studies, most (n=13) were RCTs,46–50 53 54 58–63 while the remaining (n=5) were non-RCTs.51 52 55–57 Most studies were peer-reviewed and published (n=17) and one was an unpublished doctoral thesis.57 A total of 32 712 infants (17 135 in intervention groups and 15 577 in control groups) were included in this review. Studies were undertaken in 11 different countries, including Nigeria (n=5),49 52 57 61 62 Guatemala (n=3),47 59 60 Kenya (n=2),48 63 and one each in Bangladesh,55 Burkina Faso,53 Cote D’ivoire,58 India,54 Pakistan,50 South Africa,56 Vietnam51 and Zimbabwe.46 Included studies reported 19 different interventions: 9 in urban settings, 7 in rural settings and 3 were in mixed settings. All 18 studies reported outcome data on childhood immunisation coverage, out of which 12 studies reported outcome data on childhood immunisation timeliness. Included studies reported outcome data based on either home-based health records/EPI Immunisation cards (n=7) or facility-based immunisation registers (n=11). The median sample size of included studies was 720 children and IQR is 1724.

    View this table:
    Table 1

    Description of included studies and reported outcomes

    Risk of bias assessment

    Of the 13 RCTs, about half (n=6, 46%) were rated as having a low risk of bias, 5 were rated as having some concerns and 2 were rated as having high risk of bias (online supplemental file 4A). Most of the included RCTs (n=10; 77%) were rated as having low risk of bias arising from the randomisation process whereas the remaining RCTs were rated as having some concerns for this domain. Most RCTs (n=12; 92%) had a low risk of bias in the selection of the reported result. Based on weighted risk using trials’ sample size, 30% of the included RCTs were rated as having a low risk of bias, about 60% as having some concerns and about 10% high risk of bias (online supplemental file 4B).

    Of the five non-RCTs, most (n=4; 80%) were rated as having serious or critical risk of bias whereas only one (20%) was rated with a low risk of bias in all domains (online supplemental file 5A). Of note, all included non-RCTs were rated as low risk of bias for classification of participants, deviation from intended interventions, measurement of outcomes and selection of reported results. Based on weighted risk using trials’ sample size, 15% of the included non-RCTs were rated as having a low risk of bias, 82% as having serious risk of bias and about 3% critical risk of bias (online supplemental file 5B). In general, the main causes of serious overall bias risk according to ROBINS-I assessment for non-RCTs were weaknesses in the confounding bias, selection of participants and missing data bias domains.

    Childhood immunisation coverage

    Twelve studies46 48–53 55 57 58 61 62 showed that SMS reminders significantly improved childhood immunisation coverage in children in the intervention group compared with those in the control group with usual care. Three studies showed that childhood immunisation coverage in the control group was relatively high (compared with intervention group).47 56 59 Two other studies evaluated the utility of SMS reminders in one arm of a three-arm study including compliance-linked monetary incentives in other arms.54 63 Both studies reported insignificant effectiveness for SMS reminders alone but statistically significant effect when coupled with monetary incentives.54 63 Finally, in another study, countrywide vaccine shortages precluded the evaluation of SMS reminders on overall childhood immunisation coverage.60 However, SMS reminders demonstrated statistically significant improvement in childhood immunisation timeliness.60

    Meta-analysis of data from included interventions (n=19, sample size=32 712 participants) showed that SMS reminders significantly improved childhood immunisation coverage; RR=1.16; 95% CI: 1.10 to 1.21; I2=90.4%—figure 2A. However, the predictive interval for this effect overlaps the null (0.96, 1.41), indicating some uncertainty about the distribution of effects in comparable populations. Pooled estimates using the fixed-effect model (RR=1.13; 95% CI: 1.12 to 1.15; I2=90.9%) and the random-effects REML model (RR=1.17; 95% CI: 1.09 to 1.25; I2=95.6%) were similar. Meta-analysis conducted by excluding the studies with fewer than 200 participants in either arm also produced similar results: RR=1.16; 95% CI: 1.10 to 1.23; I2=92.6%. Also, pooled estimates of studies excluding two studies with outlying sample size49 51 produced similar results: RR=1.17; 95% CI: 1.10 to 1.25; I2=90.9%. Lastly, pooled estimates excluding studies that reported only DPT-3 data showed similar results: RR=1.17; 95% CI: 1.10 to 1.23; I2=93.4%.

    Figure 2
    Figure 2

    Forest plot of pooled estimates on childhood immunisation coverage (A) and timeliness (B). Uddin et al, 2020 reported two separate interventions: Uddin, 2020 (A) for rural and Uddin, 2020 (B) for urban settings.

    In subgroup analysis, we found substantial differences in intervention effect size by country’s income status and the study’s quality and a marginal difference by study design. We found no difference in effects by study setting, outcome measure, number of SMS reminders sent or the timing of the SMS reminder—table 2. While SMS reminders were marginally effective in upper middle-income countries, they were significantly more effective in lower middle-income and low-income countries (p<0.001). Meta-regression analysis shows that only countries’ income status was a statistically significant intervention effect modifier (online supplemental file 6). A change from upper middle-income status to lower middle-income status corresponds to an increase of intervention effect of 18.1% (p=0.002) while a change from upper middle-income status to low-income status corresponds to 27.1% increase of intervention effect (p=0.006). Funnel plot presents graphical diagnostics of small study effects based on subjective visual inspection. Symmetric location of individual plots in the funnel plots indicates absence of publication bias. Graphical assessment of the funnel plot suggests absence of publication bias (online supplemental file 7). Objective assessment of publication bias using the Harbord test also indicated there is no evidence of publication bias (p value=0.2088) (online supplemental file 8).

    View this table:
    Table 2

    Subgroup analysis of SMS reminders effectiveness on improving childhood immunisation coverage

    Childhood immunisation timeliness

    Of the 12 included studies that evaluated the effect of SMS reminders on childhood immunisation timeliness, 10 studies demonstrated that SMS reminders significantly improve timely receipt of vaccines in children in the intervention group compared with those in the control group with usual care. Meta-analysis of data from included interventions (n=12, sample size=25 257 participants) showed that SMS reminders significantly improved timely receipt of childhood vaccines; RR=1.21; 95% CI: 1.12 to 1.30; I2=87.3%—figure 2B. However, the predictive interval for this effect overlaps the null (0.93, 1.56), indicating some uncertainty about the distribution of effects in similar populations.

    Although pooled meta-estimates using the fixed-effect model showed similar statistically significant results (RR=1.22; 95% CI: 1.19 to 1.25; I2=87.8%), the pooled estimates obtained using the random-effects REML model were not statistically significant (RR=1.33; 95% CI: 0.98 to 1.81; I2=99.4%). Meta-analysis conducted by excluding the studies with fewer than 200 participants in either arm also produced similar results: RR=1.18; 95% CI: 1.13 to 1.23; I2=65.2%. Also, pooled estimates of studies excluding two studies with outlying sample size49 51 equally produced similar results; RR=1.23; 95% CI: 1.10 to 1.38; I2=89.3%. Lastly, pooled estimates excluding studies that reported only DPT-3 data showed similar results; RR=1.20; 95% CI: 1.12 to 1.28; I2=72.9%.

    Subgroups analysis showed minor variation in the effectiveness of SMS reminders in improving timely receipt of childhood vaccines across different country income status, settings, study designs, outcome measure, timeliness cut-offs and study quality (table 3). However, the timely receipt of childhood vaccines was significantly improved when more than two SMS reminders were sent for childhood immunisation appointment versus when one or two SMS reminders were sent, and this difference was statistically significant (p=0.040). Also, SMS reminders significantly improved timeliness for scheduled immunisation day versus for later appointment days (p=0.024). Meta-regression analysis shows that only the number of SMS reminders sent was a statistically significant modifier of the intervention effect—online supplemental file 9. Increasing the number of SMS reminders to more than two SMS reminders improves the intervention effect by 24.0% (p=0.008). Funnel plot presents graphical diagnostics of small study effects based on subjective visual inspection. Symmetric location of individual plots in the funnel plots indicates absence of publication bias. Graphical assessment of the funnel plot suggests absence of publication bias (online supplemental file 9). Objective assessment of publication bias using the Harbord test also indicated that there is no evidence of publication bias (p value=0.7125) (online supplemental file 9).

    View this table:
    Table 3

    Subgroup analysis of SMS reminders effectiveness on improving childhood immunisation timeliness

    Body of evidence

    In this review, the body of evidence for the effectiveness of SMS reminders for improving childhood immunisation coverage was graded as moderate (online supplemental file 12). The risk of bias was serious and there was high heterogeneity among studies which downgraded the quality of the body. The body for evidence for the effectiveness of SMS reminders for improving childhood immunisation timeliness was graded as high considering several factors including strong association and dose response gradient (online supplemental file 12).

    Discussion

    Vaccination is a critical priority for LMICs. Existing systems for delivery of childhood vaccines in several LMICs have struggled to reach the critical target required to forestall outbreaks of vaccine preventable diseases.37 85 Efforts to raise global childhood immunisation levels require a strong focus on the countries where the highest numbers of unvaccinated children live while not neglecting countries where children are most likely to miss out on childhood immunisation.86 This meta-analysis of 18 articles presents the best available evidence on the effectiveness of SMS reminders on improving childhood immunisation coverage and timeliness in children less than 2 years in LMICs.

    Our pooled estimate showed that SMS reminders have the potential to improve childhood immunisation coverage in children. Even though the included studies showed substantial heterogeneity, the magnitude of the effects are large and uniformly in the positive direction. We did not find any change in level of heterogeneity when we assessed the intervention effect by the number of SMS reminders sent, the timing of the last SMS reminder, the study setting and the study design. However, subanalysis of intervention effect by countries’ income status showed decreased heterogeneity. The precise effect of SMS reminders on vaccination rates is likely to be influenced by income status of the country. We found relatively lower effectiveness of SMS reminders in upper middle-income countries although these countries have better childhood immunisation infrastructure than lower middle-income and low-income countries. Upper middle-income countries are likely to have high vaccination rates and thus limited potential for SMS reminders to improve the childhood immunisation rates further (ceiling effect).47 56 59 60 63 However, our finding contradicts a small number of studies that have shown SMS reminders to be effective in high income countries.13 34 Our study findings are generally comparable with findings from similar studies,13 87–89 though Yunusa et al suggests that both phone call reminders and SMS combined with voice message reminders are more effective than SMS reminders alone. However, compared with recent studies, our study specifically focused on SMS reminders,87 89 exclusively focused on LMICs,13 88 is a comprehensive update on SMS reminders,13 88 and employed meta-analysis to obtained an estimate of overall pooled effect.88

    We also found that SMS reminders can improve childhood immunisation timeliness, although included studies showed substantial heterogeneity. Observed heterogeneity could be due to the methodological and clinical differences between the studies, although the magnitude and direction of the intervention effects mitigates this concern.38 Substantial heterogeneity remained when we assessed the effect by country income status, study design, study setting, number of SMS reminders sent and timing of last SMS reminder, but it dropped when we limited the analysis to non-RCTs and in participants that received one or two SMS reminders. The precise effect of these interventions is likely to be greatly influenced by the number of SMS reminders sent. Sending more than two SMS reminders and sending the last SMS reminders at least 24–48 hours before the scheduled immunisation day was found to be most effective. These findings suggests that habituation—the propensity to ignore messages because of high frequency—is not a problem and that a reminder sent at least 24 hours before the scheduled immunisation day allows sufficient time for the mother/caregiver to prepare for the appointment. The effectiveness of SMS reminders in improving timely childhood vaccination was consistent across income status indicating the value for this intervention across multiple settings. Similar to our study findings for childhood immunisation coverage, our study findings are highly comparable to similar studies that specifically explored effectiveness of SMS reminders on childhood immunisation timeliness in LMICs.87

    Our findings are consistent with findings of other reviews that have assessed the effectiveness of SMS reminders on the childhood vaccination uptake,13 18 88 90 91 adherence to tuberculosis treatment,92 focused antenatal care visit and skilled birth attendance27 and attending clinic appointments.10 28 Likewise, improvement in timely vaccination due to SMS reminders is consistent with the findings of other reviews.88 A recent Cochrane systematic review comparing different types of patient reminders, however, noted that telephone reminders were more effective than SMS reminders.14 While telephone reminders may be more advantageous in reaching a population with limited or no education, the debate on which reminder is more cost-effective is inconclusive.47 93–98 Given that several studies have demonstrated mothers’ preference for phone call reminders over SMS reminders, particularly in populations with low literacy and resource-constrained settings,17 19 21 47 it is important to further explore in future studies if and how SMS reminders can be combined with mobile phone reminders (multiple modes of reminders) to obtain optimal outcomes.

    Although this study demonstrated that SMS reminder is an effective tool for increasing vaccination uptake and timeliness in LMICs, a number of critical factors highlighted from included studies must be considered before implementing this intervention. Several mothers in included studies did not have personal mobile phones.52 62 63 Compared with urban setting, rural settings have high proportion of families sharing a single phone, which proves difficult to know if the message is going to the right person.17 21 This could considerably impact the roll-out of this intervention at scale. Furthermore, implementing this intervention could exacerbate existing health inequalities due to education and wealth.17 Additionally, mobile phone interventions require extensive infrastructure for mobile communication and mothers to have the facilities for charging their phones which might not always be the case, especially in rural communities.49 57 Also, SMS reminders depend on mothers being able to read SMS messages. Mothers in some studies have indicated preference for mobile phone calls to SMS as this allows two-way communication for questions and clarifications.46 Furthermore, it is difficult to know if the SMS reminders sent were received and/or read, as one-way texting cellular companies may only record whether a message was sent, not received or read.59 60 Finally, forgetting vaccination appointment or ignoring child vaccination schedules are often not the main hurdle to accessing care. SMS reminders do not help address other prominent challenges such as accessibility (transportation, transport costs), attitude of healthcare workers, availability of vaccines, awareness of the importance of vaccination and outright rejection of vaccination.49

    Strengths and limitations

    Our approach to this review has several strengths. We included RCTs and non-RCTs, searched multiple databases including grey literature and trial registries, and considered studies in any location or language. Therefore, we believe that we have assembled the widest possible body of relevant knowledge. In addition, we rigorously adhered to widely accepted guidelines for conducting and reporting systematic reviews. However, our findings are not without limitations. Risk of bias assessment indicated some concerns for risk of bias from the randomisation process in about a quarter of included RCTs. Likewise, we cannot rule out the possibility of confounding in the included non-RCTs. Furthermore, patients and/or clinical assessors in over half of included RCTs were not blinded, although a recent study showed no difference in estimated treatment effect between trials with and without blinded patients, healthcare providers or outcome assessors.99 We included studies using two outcome measures; DPT-3 coverage and overall childhood immunisation coverage, which could have had disparate impact on the overall pooled estimate. Additionally, although we explored source of heterogeneity and differences in effect size, subgroup analyses must be interpreted with caution due to small sample size.100

    Implications for practice and research

    Consistent with a growing body of literature, our study suggests the potential of mobile interventions including SMS reminders to boost the stagnating childhood immunisation coverage in LMICs as well as the timeliness of vaccination. More generally, digital communication technologies, including mHealth, have promising impact on healthcare. Hence, it is essential to support further innovative mHealth initiatives from all stakeholders. As more LMICs prioritise core e-government systems to facilitate social, health and economic activities,23 governments and development partners need to consider building vaccination e-registries to leverage this digitalisation trend. Our review indicates that more qualitative studies are needed to understand the nuanced social and cultural details such as the optimal number of reminders, ideal time for sending reminders and content of reminder message for operational optimisation of SMS reminders. Finally, our review also suggests the need for more qualitative and quantitative studies to evaluate the effectiveness of a combination of multiple modes of reminders (eg, phone call and SMS reminders; or voice message and SMS reminders) on childhood immunisation coverage and timeliness especially in rural areas or low-literacy populations.

    Conclusion

    Meta-analysis of several LMIC studies has shown that SMS reminders can be effective at improving childhood vaccination uptake and timeliness. It is reasonable that in resource-constrained settings, simple and cost-effective mHealth interventions such as SMS reminders should be implemented in the healthcare system to improve child health outcomes. This is even more important in LMICs where increasing childhood immunisation coverage is a public health priority.

    Data availability statement

    Extracted data are available, on request, from the corresponding author.

    Ethics statements

    Acknowledgments

    The authors gratefully acknowledge the support of Dr Ujunwa Agu in editing the draft manuscript.

    References

    1. ↵
      1. World Health Organization
      . Immunization [Internet]., 2020. The global health Observatory. Available: https://www.who.int/data/gho/data/themes/immunization#
    2. ↵
      1. World Health Organization
      . National immunization coverage Scorecards estimates for 2018. Geneva, 2019.
    3. ↵
      1. Munk C,
      2. Portnoy A,
      3. Suharlim C, et al
      . Systematic review of the costs and effectiveness of interventions to increase infant vaccination coverage in low- and middle-income countries. BMC Health Serv Res 2019;19:1–10.doi:10.1186/s12913-019-4468-4
      OpenUrlCrossRef
    4. ↵
      1. Eze P,
      2. Agu UJ,
      3. Aniebo CL, et al
      . Factors associated with incomplete immunisation in children aged 12–23 months at subnational level, Nigeria: a cross-sectional study. BMJ Open 2021;11:e047445. doi:10.1136/bmjopen-2020-047445
    5. ↵
      1. World Health Organization
      . Global vaccine action plan, 2019 regional reports on progress on progress toward GVAP–RVAP goals Annex to the GVAP review and lessons learned report. Geneva, 2019.
    6. ↵
      1. Hasan MM,
      2. Magalhaes RJS,
      3. Ahmed S, et al
      . Meeting the global target in reproductive, maternal, newborn, and child health care services in low- and middle-income countries. Glob Health Sci Pract 2020;8:654–65.doi:10.9745/GHSP-D-20-00097
      OpenUrlAbstract/FREE Full Text
    7. ↵
      1. Madhi SA,
      2. Rees H
      . Special focus on challenges and opportunities for the development and use of vaccines in Africa. Hum Vaccin Immunother 2018;14:2335–9.doi:10.1080/21645515.2018.1522921
      OpenUrlPubMed
    8. ↵
      1. Bangura JB,
      2. Xiao S,
      3. Qiu D, et al
      . Barriers to childhood immunization in sub-Saharan Africa: a systematic review. BMC Public Health 2020;20.doi:10.1186/s12889-020-09169-4
    9. ↵
      1. Steinglass R
      . Routine immunization: an essential but wobbly platform. Glob Health Sci Pract 2013;1:295–301.doi:10.9745/GHSP-D-13-00122pmid:http://www.ncbi.nlm.nih.gov/pubmed/25276544
      OpenUrlFREE Full Text
    10. ↵
      1. Boksmati N,
      2. Butler-Henderson K,
      3. Anderson K, et al
      . The effectiveness of SMS reminders on appointment attendance: a meta-analysis. J Med Syst 2016;40:1–10.doi:10.1007/s10916-016-0452-2
      OpenUrl
    11. ↵
      1. Ozawa S,
      2. Clark S,
      3. Portnoy A, et al
      . Return on investment from childhood immunization in low- and middle-income countries, 2011–20. Health Aff 2016;35:199–207.doi:10.1377/hlthaff.2015.1086
      OpenUrlAbstract/FREE Full Text
    12. ↵
      1. Feikin DR,
      2. Flannery B,
      3. Hamel MJ
      . Vaccines for children in low- and middle-income countries. In: Black RE, Laxminarayan R, Temmerman M, eds. Reproductive, maternal, newborn, and child health: disease control priorities. 3rd edn. Washington DC: World Bank, 2016.
    13. ↵
      1. Mekonnen ZA,
      2. Gelaye KA,
      3. Were MC, et al
      . Effect of mobile text message reminders on routine childhood vaccination: a systematic review and meta-analysis. Syst Rev 2019;8:154. doi:10.1186/s13643-019-1054-0pmid:http://www.ncbi.nlm.nih.gov/pubmed/31253186
      OpenUrlPubMed
    14. ↵
      1. Jacobson Vann JC,
      2. Jacobson RM,
      3. Coyne-Beasley T, et al
      . Patient reminder and recall interventions to improve immunization rates. Cochrane Database Syst Rev 2018;1:CD003941. doi:10.1002/14651858.CD003941.pub3pmid:http://www.ncbi.nlm.nih.gov/pubmed/29342498
      OpenUrlCrossRefPubMed
    15. ↵
      1. Kolff CA,
      2. Scott VP,
      3. Stockwell MS
      . The use of technology to promote vaccination: a social ecological model based framework. Hum Vaccin Immunother 2018;14:1636–46.doi:10.1080/21645515.2018.1477458
      OpenUrlPubMed
    16. ↵
      1. Linkins RW,
      2. Dini EF,
      3. Watson G, et al
      . A randomized trial of the effectiveness of computer-generated telephone messages in increasing immunization visits among preschool children. Arch Pediatr Adolesc Med 1994;148:908–14.doi:10.1001/archpedi.1994.02170090022002pmid:http://www.ncbi.nlm.nih.gov/pubmed/8075732
      OpenUrlCrossRefPubMedWeb of Science
    17. ↵
      1. Gibson DG,
      2. Tamrat T,
      3. Mehl G
      . The state of digital interventions for demand generation in low- and middle-income countries: considerations, emerging approaches, and research gaps. Glob Heal Sci Pract 2018:49–60.
    18. ↵
      1. Atkinson KM,
      2. Wilson K,
      3. Murphy MSQ, et al
      . Effectiveness of digital technologies at improving vaccine uptake and series completion – a systematic review and meta-analysis of randomized controlled trials. Vaccine 2019;37:3050–60.doi:10.1016/j.vaccine.2019.03.063
      OpenUrl
    19. ↵
      1. Domek GJ,
      2. Contreras-Roldan IL,
      3. Asturias EJ, et al
      . Characteristics of mobile phone access and usage in rural and urban Guatemala: assessing feasibility of text message reminders to increase childhood immunizations. mHealth 2018;4:9–10.doi:10.21037/mhealth.2018.03.05
      OpenUrl
    20. ↵
      1. Kannisto KA,
      2. Koivunen MH,
      3. Anneli M
      . Use of Mobile Phone Text Message Reminders in Health Care Services : A Narrative Literature Review Corresponding Author . J Meidcal Internet Res 2014;16:1–14.
      OpenUrl
    21. ↵
      1. Kazi AM
      . The role of mobile phone-based interventions to improve routine childhood immunisation coverage. Lancet Glob Health 2017;5:e377–8.doi:10.1016/S2214-109X(17)30088-8
      OpenUrl
    22. ↵
      1. Radcliffe D
      . Mobile in Sub-Saharan Africa: can world’s fastest-growing mobile region keep it up? [Internet], 2018. Mobility. Available: https://www.zdnet.com/article/mobile-in-sub-saharan-africa-can-worlds-fastest-growing-mobile-region-keep-it-up/
    23. ↵
      1. World Bank Group
      . Digital dividends overview. In: World development report. 2016. Washington DC: World Bank, 2016.
    24. ↵
      1. DeSouza SI,
      2. Rashmi MR,
      3. Vasanthi AP, et al
      . Mobile phones: the next step towards healthcare delivery in rural India? PLoS One 2014;9:e104895. doi:10.1371/journal.pone.0104895pmid:http://www.ncbi.nlm.nih.gov/pubmed/25133610
      OpenUrlCrossRefPubMed
    25. ↵
      1. Eze E,
      2. Gleasure R,
      3. Heavin C
      . Mobile health solutions in developing countries: a stakeholder perspective. Health Syst 2020;9:179–201.doi:10.1080/20476965.2018.1457134
      OpenUrl
    26. ↵
      1. Lee S,
      2. Cho Y-min,
      3. Kim S-Y
      . Mapping mHealth (mobile health) and mobile penetrations in sub-Saharan Africa for strategic regional collaboration in mHealth scale-up: an application of exploratory spatial data analysis. Global Health 2017;13:1–11.doi:10.1186/s12992-017-0286-9
      OpenUrlPubMed
    27. ↵
      1. Wagnew F,
      2. Dessie G,
      3. Alebel A, et al
      . Does short message service improve focused antenatal care visit and skilled birth attendance? A systematic review and meta-analysis of randomized clinical trials. Reprod Health 2018;15:191. doi:10.1186/s12978-018-0635-z
    28. ↵
      1. Guy R,
      2. Hocking J,
      3. Wand H, et al
      . How effective are short message service reminders at increasing clinic attendance? A meta-analysis and systematic review. Health Serv Res 2012;47:614–32.doi:10.1111/j.1475-6773.2011.01342.xpmid:http://www.ncbi.nlm.nih.gov/pubmed/22091980
      OpenUrlCrossRefPubMedWeb of Science
    29. ↵
      1. Demena BA,
      2. Artavia-Mora L,
      3. Ouedraogo D, et al
      . A systematic review of mobile phone interventions (SMS/IVR/Calls) to improve adherence and retention to antiretroviral treatment in Low-and middle-income countries. AIDS Patient Care STDS 2020;34:59–71.doi:10.1089/apc.2019.0181pmid:http://www.ncbi.nlm.nih.gov/pubmed/32049555
      OpenUrlPubMed
    30. ↵
      1. Ershad Sarabi R,
      2. Sadoughi F,
      3. Jamshidi Orak R, et al
      . The effectiveness of mobile phone text messaging in improving medication adherence for patients with chronic diseases: a systematic review. Iran Red Crescent Med J 2016;18:e25183. doi:10.5812/ircmj.25183pmid:http://www.ncbi.nlm.nih.gov/pubmed/27437126
      OpenUrlPubMed
    31. ↵
      1. Menzies R,
      2. Heron L,
      3. Lampard J, et al
      . A randomised controlled trial of SMS messaging and calendar reminders to improve vaccination timeliness in infants. Vaccine 2020;38:3137–42.doi:10.1016/j.vaccine.2020.02.045
      OpenUrl
    32. ↵
      1. Hofstetter AM,
      2. Vargas CY,
      3. Camargo S, et al
      . Impacting delayed pediatric influenza vaccination: a randomized controlled trial of text message reminders. Am J Prev Med 2015;48:392–401.doi:10.1016/j.amepre.2014.10.023pmid:http://www.ncbi.nlm.nih.gov/pubmed/25812465
      OpenUrlCrossRefPubMed
    33. ↵
      1. Rand CM,
      2. Brill H,
      3. Albertin C, et al
      . Effectiveness of centralized text message reminders on human papillomavirus immunization coverage for publicly insured adolescents. J Adolesc Health 2015;56:S17–20.doi:10.1016/j.jadohealth.2014.10.273pmid:http://www.ncbi.nlm.nih.gov/pubmed/25863549
      OpenUrlPubMed
    34. ↵
      1. Stockwell MS,
      2. Kharbanda EO,
      3. Martinez RA, et al
      . Text4Health: impact of text message reminder-recalls for pediatric and adolescent immunizations. Am J Public Health 2012;102:e15–21.doi:10.2105/AJPH.2011.300331pmid:http://www.ncbi.nlm.nih.gov/pubmed/22390457
      OpenUrlCrossRefPubMed
    35. ↵
      1. Moher D, et al
      . Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med 2009;151:264. doi:10.7326/0003-4819-151-4-200908180-00135
    36. ↵
      1. World Bank
      . World Bank Country and Lending Groups: Country classification [Internet], 2020. Data. Available: https://datahelpdesk.worldbank.org/knowledgebase/articles/906519-world-bank-country-and-lending-groups
    37. ↵
      1. Mosser JF,
      2. Gagne-Maynard W,
      3. Rao PC, et al
      . Mapping diphtheria-pertussis-tetanus vaccine coverage in Africa, 2000-2016: a spatial and temporal modelling study. Lancet 2019;393:1843–55.doi:10.1016/S0140-6736(19)30226-0pmid:http://www.ncbi.nlm.nih.gov/pubmed/30961907
      OpenUrlCrossRefPubMed
    38. ↵
      1. Higgins J,
      2. Thomas J,
      3. Chandler J, et al
      . Cochrane Handbook for Systematic Reviews of Interventions Version 6.1 [updated September 2020] [Internet]. Cochrane, 2020. www.training.cochrane.org/handbook
    39. ↵
      1. Sterne JAC,
      2. Savović J,
      3. Page MJ, et al
      . Rob 2: a revised tool for assessing risk of bias in randomised trials. BMJ 2019;2:l4898–8.doi:10.1136/bmj.l4898
      OpenUrl
    40. ↵
      1. Sterne JAC,
      2. Hernán MA,
      3. Reeves BC, et al
      . ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ 2016;355:i4919–10.doi:10.1136/bmj.i4919
      OpenUrlFREE Full Text
    41. ↵
      1. McGuinness LA,
      2. Higgins JPT
      . Risk-of-bias visualization (robvis): an R package and shiny web APP for visualizing risk-of-bias assessments. Res Synth Methods 2021;12:55–61.doi:10.1002/jrsm.1411pmid:http://www.ncbi.nlm.nih.gov/pubmed/32336025
      OpenUrlPubMed
    42. ↵
      1. Borenstein M,
      2. Hedges LV,
      3. Higgins JPT, et al
      . A basic introduction to fixed-effect and random-effects models for meta-analysis. Res Synth Methods 2010;1:97–111.doi:10.1002/jrsm.12pmid:http://www.ncbi.nlm.nih.gov/pubmed/26061376
      OpenUrlPubMed
    43. ↵
      1. Borenstein M,
      2. Higgins JPT,
      3. Hedges LV, et al
      . Basics of meta-analysis: I2 is not an absolute measure of heterogeneity. Res Synth Methods 2017;8:5–18.doi:10.1002/jrsm.1230pmid:http://www.ncbi.nlm.nih.gov/pubmed/28058794
      OpenUrlPubMed
    44. ↵
      1. Langan D,
      2. Higgins JPT,
      3. Jackson D, et al
      . A comparison of heterogeneity variance estimators in simulated random‐effects meta‐analyses. Res Synth Methods 2019;10:83–98.doi:10.1002/jrsm.1316
      OpenUrlPubMed
    45. ↵
      1. Schünemann H,
      2. Brożek J,
      3. Guyatt G
      . Handbook for grading the quality of evidence and the strength of recommendations using the GRADE approach [Internet], 2013. Available: https://gdt.gradepro.org/app/handbook/handbook.html
    46. ↵
      1. Bangure D,
      2. Chirundu D,
      3. Gombe N, et al
      . Effectiveness of short message services reminder on childhood immunization programme in Kadoma, Zimbabwe - a randomized controlled trial, 2013. BMC Public Health 2015;15:1–8.doi:10.1186/s12889-015-1470-6
      OpenUrlCrossRefPubMed
    47. ↵
      1. Ceballos F,
      2. Hernandez MA,
      3. Olivet F, et al
      . Assessing the use of cell phones to monitor health and nutrition interventions: evidence from rural Guatemala. PLoS One 2020;15:e0240526. doi:10.1371/journal.pone.0240526pmid:http://www.ncbi.nlm.nih.gov/pubmed/33141833
      OpenUrlPubMed
    48. ↵
      1. Haji A,
      2. Lowther S,
      3. Ngan'ga Z, et al
      . Reducing routine vaccination dropout rates: evaluating two interventions in three Kenyan districts, 2014. BMC Public Health 2016;16:152–8.doi:10.1186/s12889-016-2823-5pmid:http://www.ncbi.nlm.nih.gov/pubmed/26880141
      OpenUrlPubMed
    49. ↵
      1. Kawakatsu Y,
      2. Oyeniyi Adesina A,
      3. Kadoi N, et al
      . Cost-effectiveness of SMS appointment reminders in increasing vaccination uptake in Lagos, Nigeria: a multi-centered randomized controlled trial. Vaccine 2020;38:6600–8.doi:10.1016/j.vaccine.2020.07.075pmid:http://www.ncbi.nlm.nih.gov/pubmed/32788139
      OpenUrlPubMed
    50. ↵
      1. Kazi AM,
      2. Ali M,
      3. Zubair K, et al
      . Effect of mobile phone text message reminders on routine immunization uptake in Pakistan: randomized controlled trial. JMIR Public Health Surveill 2018;4:e20. doi:10.2196/publichealth.7026pmid:http://www.ncbi.nlm.nih.gov/pubmed/29514773
      OpenUrlPubMed
    51. ↵
      1. Nguyen NT,
      2. Vu HM,
      3. Dao SD, et al
      . Digital immunization registry: evidence for the impact of mHealth on enhancing the immunization system and improving immunization coverage for children under one year old in Vietnam. Mhealth 2017;3:26.doi:10.21037/mhealth.2017.06.03pmid:http://www.ncbi.nlm.nih.gov/pubmed/28828373
      OpenUrlCrossRefPubMed
    52. ↵
      1. Oladepo O,
      2. Dipeolu IO,
      3. Oladunni O
      . Outcome of reminder text messages intervention on completion of routine immunization in rural areas, Nigeria. Health Promot Int 2020:daaa092. doi:10.1093/heapro/daaa092pmid:http://www.ncbi.nlm.nih.gov/pubmed/33057615
      OpenUrlPubMed
    53. ↵
      1. Schlumberger M,
      2. Bamoko A,
      3. Yaméogo TM
      . Positive impact on the expanded program on immunization when sending call-back SMS through a computerized immunization register, Bobo Dioulasso (Burkina Faso). Bull la Soc Pathol Exot 2015;108:349–54.
      OpenUrl
    54. ↵
      1. Seth R,
      2. Akinboyo I,
      3. Chhabra A, et al
      . Mobile phone incentives for childhood immunizations in rural India. Pediatrics 2018;141:e20173455. doi:10.1542/peds.2017-3455
    55. ↵
      1. Uddin MJ,
      2. Shamsuzzaman M,
      3. Horng L, et al
      . Use of mobile phones for improving vaccination coverage among children living in rural hard-to-reach areas and urban streets of Bangladesh. Vaccine 2016;34:276–83.doi:10.1016/j.vaccine.2015.11.024
      OpenUrlCrossRefPubMed
    56. ↵
      1. Coleman J,
      2. Black V,
      3. Thorson AE, et al
      . Evaluating the effect of maternal mHealth text messages on uptake of maternal and child health care services in South Africa: a multicentre cohort intervention study. Reprod Health 2020;17:160–9.doi:10.1186/s12978-020-01017-3pmid:http://www.ncbi.nlm.nih.gov/pubmed/33081795
      OpenUrlPubMed
    57. ↵
      1. Dipeolu IO
      . Effect of Mobile-phone reminder text messages on Mothers’ knowledge and completion of routine immunization in rural areas of Oyo State, Nigeria. University of Ibadan, 2017. https://library.adhl.africa/handle/123456789/11865
    58. ↵
      1. Dissieka R,
      2. Soohoo M,
      3. Janmohamed A, et al
      . Providing mothers with mobile phone message reminders increases childhood immunisation and vitamin A supplementation coverage in Côte d’Ivoire: a randomised controlled trial. J Public Health Africa 2019;10:56–60.doi:10.4081/jphia.2019.1032
      OpenUrl
    59. ↵
      1. Domek GJ,
      2. Contreras-Roldan IL,
      3. O'Leary ST, et al
      . Sms text message reminders to improve infant vaccination coverage in Guatemala: a pilot randomized controlled trial. Vaccine 2016;34:2437–43.doi:10.1016/j.vaccine.2016.03.065pmid:http://www.ncbi.nlm.nih.gov/pubmed/27026145
      OpenUrlCrossRefPubMed
    60. ↵
      1. Domek GJ,
      2. Contreras-Roldan IL,
      3. Bull S, et al
      . Text message reminders to improve infant immunization in Guatemala: a randomized clinical trial. Vaccine 2019;37:6192–200.doi:10.1016/j.vaccine.2019.08.046pmid:http://www.ncbi.nlm.nih.gov/pubmed/31492475
      OpenUrlPubMed
    61. ↵
      1. Ekhaguere OA,
      2. Oluwafemi RO,
      3. Badejoko B, et al
      . Automated phone call and text reminders for childhood immunisations (PRIMM): a randomised controlled trial in Nigeria. BMJ Glob Health 2019;4:e001232.doi:10.1136/bmjgh-2018-001232pmid:http://www.ncbi.nlm.nih.gov/pubmed/31139442
      OpenUrlAbstract/FREE Full Text
    62. ↵
      1. Eze GU,
      2. Adeleye OO
      . Enhancing routine immunization performance using innovative technology in an urban area of Nigeria. West Afr J Med 2015;34:3–10.pmid:http://www.ncbi.nlm.nih.gov/pubmed/26902809
      OpenUrlPubMed
    63. ↵
      1. Gibson DG,
      2. Ochieng B,
      3. Kagucia EW, et al
      . Mobile phone-delivered reminders and incentives to improve childhood immunisation coverage and timeliness in Kenya (M-SIMU): a cluster randomised controlled trial. Lancet Glob Health 2017;5:e428–38.doi:10.1016/S2214-109X(17)30072-4pmid:http://www.ncbi.nlm.nih.gov/pubmed/28288747
      OpenUrlPubMed
    64. ↵
      1. Atnafu A,
      2. Otto K,
      3. Herbst CH
      . The role of mHealth intervention on maternal and child health service delivery: findings from a randomized controlled field trial in rural Ethiopia. Mhealth 2017;3:39.doi:10.21037/mhealth.2017.08.04pmid:http://www.ncbi.nlm.nih.gov/pubmed/29184891
      OpenUrlPubMed
    65. ↵
      1. Brown VB,
      2. Oluwatosin OA
      . Socio-Demographic factors associated with childhood immunization uptake in Akinyele local government area, Oyo state, Nigeria. Afr J Med Med Sci 2012;41:161–7.pmid:http://www.ncbi.nlm.nih.gov/pubmed/23185914
      OpenUrlPubMed
    66. ↵
      1. Busso M,
      2. Cristia J,
      3. Humpage S
      . Did you get your shots? experimental evidence on the role of reminders. J Health Econ 2015;44:226–37.doi:10.1016/j.jhealeco.2015.08.005
      OpenUrl
    67. ↵
      1. Chen L,
      2. Du X,
      3. Zhang L, et al
      . Effectiveness of a smartphone APP on improving immunization of children in rural Sichuan Province, China: a cluster randomized controlled trial. BMC Public Health 2016;16.doi:10.1186/s12889-016-3549-0
    68. ↵
      1. Eze NC,
      2. Onwasigwe CN,
      3. Una AF
      . Implementation of mobile phone reminder system to improve immunisation uptake in Abakaliki, Southeast, Nigeria: its feasibility and acceptability. Asian J Med Princ Clin Pract 2018.doi:10.9734/AJMPCP/2018/44948
    69. ↵
      1. Johri M,
      2. Chandra D,
      3. Kone KG, et al
      . Social and behavior change communication interventions delivered face-to-face and by a mobile phone to strengthen vaccination uptake and improve child health in rural India: randomized pilot study. JMIR Mhealth Uhealth 2020;8:e20356.doi:10.2196/20356pmid:http://www.ncbi.nlm.nih.gov/pubmed/32955455
      OpenUrlPubMed
    70. ↵
      1. Kaewkungwal J,
      2. Apidechkul T,
      3. Jandee K, et al
      . Application of mobile technology for improving expanded program on immunization among highland minority and stateless populations in northern Thailand border. JMIR Mhealth Uhealth 2015;3:e4–14.doi:10.2196/mhealth.3704pmid:http://www.ncbi.nlm.nih.gov/pubmed/25589367
      OpenUrlPubMed
    71. ↵
      1. Murthy N,
      2. Chandrasekharan S,
      3. Prakash MP, et al
      . The impact of an mHealth voice message service (mMitra) on infant care knowledge, and practices among low-income women in India: findings from a Pseudo-Randomized controlled trial. Matern Child Health J 2019;23:1658–69.doi:10.1007/s10995-019-02805-5pmid:http://www.ncbi.nlm.nih.gov/pubmed/31584144
      OpenUrlPubMed
    72. ↵
      1. Uddin MJ,
      2. Saha NC,
      3. Islam Z, et al
      . Improving low coverage of child immunization in rural hard-to-reach areas of Bangladesh: findings from a project using multiple interventions. Vaccine 2012;30:168–79.doi:10.1016/j.vaccine.2011.11.030pmid:http://www.ncbi.nlm.nih.gov/pubmed/22108489
      OpenUrlPubMed
    73. ↵
      1. Ahlers-Schmidt CR,
      2. Chesser AK,
      3. Nguyen T, et al
      . Feasibility of a randomized controlled trial to evaluate text reminders for immunization compliance in kids (tricks). Vaccine 2012;30:5305–9.doi:10.1016/j.vaccine.2012.06.058pmid:http://www.ncbi.nlm.nih.gov/pubmed/22750044
      OpenUrlCrossRefPubMedWeb of Science
    74. ↵
      1. Niederhauser V,
      2. Johnson M,
      3. Tavakoli AS
      . Vaccines4Kids: assessing the impact of text message reminders on immunization rates in infants. Vaccine 2015;33:2984–9.doi:10.1016/j.vaccine.2015.04.069pmid:http://www.ncbi.nlm.nih.gov/pubmed/25957663
      OpenUrlPubMed
    75. ↵
      1. Abdullah S,
      2. Siddiqi DA,
      3. Shah MT, et al
      . Evaluating small conditional cash transfers to improve vaccine coverage and timeliness in Pakistan. Eur J Public Health 2020;30:364–5.doi:10.1093/eurpub/ckaa165.993
      OpenUrl
    76. ↵
      1. Thiaw C,
      2. Cooper A,
      3. Tendeng C, et al
      . Routine delivery of vitamin A supplementation at six months in Senegal using SMS reminder messages. Eur J Nutr Food Saf 2015;5:804–5.doi:10.9734/EJNFS/2015/21100
      OpenUrl
    77. ↵
      1. Gatuha G,
      2. Jiang T
      . KenVACS: improving vaccination of children through cellular network technology in developing countries. IJIKM 2015;10:037–46.doi:10.28945/2208
      OpenUrl
    78. ↵
      1. Kraszewski S
      . Improving immunisation uptake in babies and toddlers. Practice Nursing 2017;28:300–6.doi:10.12968/pnur.2017.28.7.300
      OpenUrl
    79. ↵
      1. Faujdar DS,
      2. Sahay S,
      3. Singh T, et al
      . Field testing of a digital health information system for primary health care: a quasi-experimental study from India. Int J Med Inform 2020;141:104235. doi:10.1016/j.ijmedinf.2020.104235
    80. ↵
      1. Kaewkungwal J,
      2. Singhasivanon P,
      3. Khamsiriwatchara A, et al
      . Application of smart phone in "Better Border Healthcare Program": A module for mother and child care. BMC Med Inform Decis Mak 2010;10.doi:10.1186/1472-6947-10-69
    81. ↵
      1. Coleman J,
      2. Xiong K
      . Gauging the Impact of MomConnect on Maternal Health Service Utilisation by Women and their infants in Johannesburg, South Africa [Internet], 2017. Available: http://www.measureevaluation.org/sifsa
    82. ↵
      1. Wakadha H,
      2. Chandir S,
      3. Were EV, et al
      . The feasibility of using mobile-phone based SMS reminders and conditional cash transfers to improve timely immunization in rural Kenya. Vaccine 2013;31:987–93.doi:10.1016/j.vaccine.2012.11.093
      OpenUrlCrossRefPubMedWeb of Science
    83. ↵
      1. Garcia-Dia MJ,
      2. Fitzpatrick JJ,
      3. Madigan EA, et al
      . Using text reminder to improve childhood immunization adherence in the Philippines. CIN - Comput Informatics Nurs 2017;35:212–8.doi:10.1097/CIN.0000000000000307
      OpenUrl
    84. ↵
      1. Shinde K,
      2. Rani U,
      3. Kumar PN
      . Assessing the effectiveness of immunization reminder system among nursing mothers of South India. Res J Pharm Technol 2018;11:1761––7.doi:10.5958/0974-360X.2018.00327.X
      OpenUrl
    85. ↵
      1. Local Burden of Disease Vaccine Coverage Collaborators
      . Mapping routine measles vaccination in low- and middle-income countries. Nature 2021;589:415–9.doi:10.1038/s41586-020-03043-4pmid:http://www.ncbi.nlm.nih.gov/pubmed/33328634
      OpenUrlPubMed
    86. ↵
      1. United Nations Children Fund (UNICEF)
      . Immunization [Internet]. UNICEF Data: Monitoring the situation of children and women, 2020. Available: https://data.unicef.org/topic/child-health/immunization/
    87. ↵
      1. Yunusa U,
      2. Garba SN,
      3. Umar AB, et al
      . Mobile phone reminders for enhancing uptake, completeness and timeliness of routine childhood immunization in low and middle income countries: a systematic review and meta-analysis. Vaccine 2021;39:209–21.doi:10.1016/j.vaccine.2020.11.043
      OpenUrlPubMed
    88. ↵
      1. Manakongtreecheep K
      . SMS-reminder for vaccination in Africa: research from published, unpublished and grey literature. Pan Afr Med J 2017;27:23.doi:10.11604/pamj.supp.2017.27.3.12115pmid:http://www.ncbi.nlm.nih.gov/pubmed/29296158
      OpenUrlPubMed
    89. ↵
      1. Oyo-Ita A,
      2. Wiysonge CS,
      3. Oringanje C, et al
      . Interventions for improving coverage of childhood immunisation in low- and middle-income countries. Cochrane Database Syst Rev 2016;9(Suppl 1).doi:10.1002/14651858.CD008145.pub3
    90. ↵
      1. Linde DS,
      2. Korsholm M,
      3. Katanga J, et al
      . One-way SMS and healthcare outcomes in Africa: systematic review of randomised trials with meta-analysis. PLoS One 2019;14:e0217485. doi:10.1371/journal.pone.0217485pmid:http://www.ncbi.nlm.nih.gov/pubmed/31170176
      OpenUrlPubMed
    91. ↵
      1. Kim SS,
      2. Patel M,
      3. Hinman A
      . Use of m-Health in polio eradication and other immunization activities in developing countries. Vaccine 2017;35:1373–9.doi:10.1016/j.vaccine.2017.01.058
      OpenUrlPubMed
    92. ↵
      1. Gashu KD,
      2. Gelaye KA,
      3. Mekonnen ZA, et al
      . Does phone messaging improves tuberculosis treatment success? A systematic review and meta-analysis. BMC Infect Dis 2020;20:1–13.doi:10.1186/s12879-020-4765-x
      OpenUrlCrossRefPubMed
    93. ↵
      1. Hasvold PE,
      2. Wootton R
      . Use of telephone and SMS reminders to improve attendance at hospital appointments: a systematic review. J Telemed Telecare 2011;17:358–64.doi:10.1258/jtt.2011.110707pmid:http://www.ncbi.nlm.nih.gov/pubmed/21933898
      OpenUrlCrossRefPubMedWeb of Science
    94. ↵
      1. Bracken K,
      2. Keech A,
      3. Hague W, et al
      . Telephone call reminders did not increase screening uptake more than SMS reminders: a recruitment study within a trial. J Clin Epidemiol 2019;112:45–52.doi:10.1016/j.jclinepi.2019.04.009
      OpenUrl
    95. ↵
      1. Hallsworth M,
      2. Berry D,
      3. Sanders M, et al
      . Stating appointment costs in SMS reminders reduces missed Hospital appointments: findings from two randomised controlled trials. PLoS One 2015;10:e0137306. doi:10.1371/journal.pone.0137306pmid:http://www.ncbi.nlm.nih.gov/pubmed/26366885
      OpenUrlPubMed
    96. ↵
      1. Junod Perron N,
      2. Dao MD,
      3. Righini NC, et al
      . Text-messaging versus telephone reminders to reduce missed appointments in an academic primary care clinic: a randomized controlled trial. BMC Health Serv Res 2013;13:125. doi:10.1186/1472-6963-13-125pmid:http://www.ncbi.nlm.nih.gov/pubmed/23557331
      OpenUrlCrossRefPubMed
    97. ↵
      1. Gurol-Urganci I,
      2. de Jongh T,
      3. Vodopivec-Jamsek V, et al
      . Mobile phone messaging reminders for attendance at healthcare appointments. Cochrane Database Syst Rev 2013;2013:CD007458. doi:10.1002/14651858.CD007458.pub3pmid:http://www.ncbi.nlm.nih.gov/pubmed/24310741
      OpenUrlPubMed
    98. ↵
      1. Chen Z-W,
      2. Fang L-Z,
      3. Chen L-Y, et al
      . Comparison of an SMS text messaging and phone reminder to improve attendance at a health promotion center: a randomized controlled trial. J Zhejiang Univ Sci B 2008;9:34–8.doi:10.1631/jzus.B071464pmid:http://www.ncbi.nlm.nih.gov/pubmed/18196610
      OpenUrlCrossRefPubMedWeb of Science
    99. ↵
      1. Moustgaard H,
      2. Clayton GL,
      3. Jones HE, et al
      . Impact of blinding on estimated treatment effects in randomised clinical trials: meta-epidemiological study. BMJ 2020;368:l6802.doi:10.1136/bmj.l6802pmid:http://www.ncbi.nlm.nih.gov/pubmed/31964641
      OpenUrlAbstract/FREE Full Text
    100. ↵
      1. Sylla F,
      2. Moreau C,
      3. Andro A
      . A systematic review and meta-analysis of the consequences of female genital mutilation on maternal and perinatal health outcomes in European and African countries. BMJ Glob Health 2020;5.doi:10.1136/bmjgh-2020-003307pmid:http://www.ncbi.nlm.nih.gov/pubmed/33380410
      OpenUrlPubMed

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Footnotes

    • Handling editor Seye Abimbola

    • Contributors PE and YA conceptualised the study and designed the protocol, with feedback from LOL. PE and LOL independently conducted the search, screening, data extraction, and assessment of bias and quality of reporting. PE conducted the meta-analysis and drafted the manuscript. LOL and YA reviewed the draft, provided critical review, and read and approved the final manuscript. The corresponding author, as guarantor, accepts full responsibility for the finished article, has access to the data and controlled the decision to publish. The corresponding author attests that all listed authors meet the authorship criteria and that no others meeting the criteria have been omitted.

    • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

    • Competing interests None declared.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.