Article Text
PDF
Abstract
Introduction India has become a lighthouse for large-scale digital innovation in the health sector, particularly for front-line health workers (FLHWs). However, among scaled digital health solutions, ensuring sustainability remains elusive. This study explores the factors underpinning scale-up of digital health solutions for FLHWs in India, and the potential implications of these factors for sustainability.
Methods We assessed five FLHW digital tools scaled at the national and/or state level in India. We conducted in-depth interviews with implementers, technology and technical partners (n=11); senior government stakeholders (n=5); funders (n=1) and evaluators/academics (n=3). Emergent themes were grouped according to a broader framework that considered the (1) digital solution; (2) actors; (3) processes and (4) context.
Results The scale-up of digital solutions was facilitated by their perceived value, bounded adaptability, support from government champions, cultivation of networks, sustained leadership and formative research to support fit with the context and population. However, once scaled, embedding digital health solutions into the fabric of the health system was hampered by challenges related to transitioning management and ownership to government partners; overcoming government procurement hurdles; and establishing committed funding streams in government budgets. Strong data governance, continued engagement with FLHWs and building a robust evidence base, while identified in the literature as critical for sustainability, did not feature strongly among respondents. Sustainability may be less elusive once there is more consensus around the roles played between national and state government actors, implementing and technical partners and donors.
Conclusion The use of digital tools by FLHWs offers much promise for improving service delivery and health outcomes in India. However, the pathway to sustainability is bespoke to each programme and should be planned from the outset by investing in people, relationships and service delivery adjustments to navigate the challenges involved given the dynamic nature of digital tools in complex health systems.
- health policy
- health systems
- qualitative study
- public Health
Data availability statement
Data are available on request. Deidentified participant data are available on request from NSS (neha.singh@lshtm.ac.uk).
This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.
Statistics from Altmetric.com
Key questions
What is already known?
Hundreds of digital health interventions have been piloted in low-income and middle-income countries, though few have been successfully scaled.
Among those that have progressed beyond pilot initiatives to attain scale, efforts to achieve sustainability remains elusive, particularly with regard to integration into routine health services delivery, independence from donor funding, interoperability and governance.
What are the new findings?
To successfully sustain a scaled up digital tool, it is imperative for all stakeholders, in particular governments and donors, to have an entire supportive ecosystem in place that addresses the dynamics between aspects of the digital solution, actor relationships, implementation processes and key contextual factors, with strong government leadership to align all these pieces.
What do the new findings imply?
Our findings challenge the notion that digital health programmes progress linearly from pilot to scale to sustainability, and instead explore the elements and tensions between scale and sustainability for digital health.
With significant resources spent each year on digital health solutions that are never scaled or sustained, it is imperative that we build the evidence base on factors that lead to success in sustaining innovations in the digital health space.
Introduction
Throughout the last decade, over 600 digital health pilot strategies and programmes have been introduced globally.1 Despite the proliferation of digital health programmes, evidence on their effectiveness is limited.2–4 Moreover, governments have found it challenging to select, scale-up, and integrate solutions into existing national systems.5
Scale-up refers to the increase of a programme’s reach from a pilot population or experimental region to greater numbers of people over wider geographical areas.6 Studies and frameworks examining scaling up have emerged over time with considerable commonality.7–10 Sustainability of a programme is the ‘capability of being maintained at a certain rate or level.’11 When applied to health programmes and policies, the concept of sustainability involves the continuation of inputs (eg, funding or programme activities), outputs (eg, health benefits) and a process of adaption in response to emerging needs of the system.12 It is an emergent field of enquiry, with numerable frameworks from different disciplines and noted challenges in assessment.10 12–16
To date, only select digital health programmes for front-line health workers (FLHWs) have scaled,17 18 including a few in India.19–21 The successful scale-up of these programmes raises important questions about the micro (eg, negotiations between individuals), meso (organisational processes and systems) and macro (national policy and wider context) level factors that underpin decision making on which initiatives get adopted and with what evidence. Major factors influencing scale-up have been synthesised to create our conceptual framework (discussed below, figure 1).10 22–24 The features that influence sustainability overlap considerably with those for scale-up, but shift focus from the intervention itself to the health system’s ability to plan, organise, adapt and communicate, and the broader fiscal and political environment.25 26
Conceptual framework for evaluating the scale-up and sustainability of digital solutions for front-line health workers.
To date, very few studies27 28 have aimed to understand processes underpinning scaled digital health tools in India or elsewhere to reflect on sustainability . Emerging literature, largely from high-income settings, sheds light on the complex and multi-dimensional nature of scaling digital solutions for health,13 29–31 as well as how the social, political and historical contexts where digital tools are implemented plays a key role in how solutions are embedded, adapted and potentially sustained within a health system.32 33
This study aims to answer ‘Why have some digital health programmes scaled, and others not, and what are the implications for sustainability in India?’. To understand the factors that underpin the scale-up and sustainability of different typologies of digital health solutions in India, we examine five FLHW digital tools ranging from direct-to-FLHW programmes to real-time data capture and decision-support tools. This analysis will benefit actors involved in the development of digital tools for FLHWs in India and other settings by illuminating the multifaceted inputs and processes involved in scale-up and by providing a framework for thinking about how the scale-up process can support or hinder the tool’s chances of sustainability. Drawing case studies from India is particularly fruitful because it is the site of several of the world’s largest digital programmes for FLHWs and one of the few low-income or middle-income countries that appear to be sustaining these tools.
Methods
Conceptual framework
We developed a conceptual framework to study what shapes scale-up and sustainability of digital tools for FLHWs (figure 1), reflecting broader literature on the subject.6 8 10 13 31 34–37
Our framework was adapted from the following existing conceptual frameworks by taking into account variables relevant to the implementation of digital tools in resource-constrained settings: (1) Greenhalgh et al’s framework taking into account complexity of scaling-up technology-supported programmes13; (2) Gericke et al’s framework which has been applied to assess the scale-up of mHealth innovations in Malawi and Zambia31; (3) Spicer et al’s framework based on studying scalable health innovations’ attributes in Ethiopia, India and Nigeria10 and (4) Gilson and Walt’s policy triangle.38 Doing so enabled us to simplify relevant variables into four themes: (1) digital solution characteristics, (2) actor roles and relationships, (3) implementation processes and (4) context. While drawing from existing frameworks for scaling and sustaining health interventions more generally, our work particularly takes into account specific ‘hardware’(eg, cloud storage) and ‘software’ (eg, technological partnerships) required to scale and sustain digital health solutions.
Study setting
In 2011, India’s population was 1.2 billion, with nearly three-quarters (74%) being literate.39 Mobile phone access in India has rapidly increased, with the 2015–2016 National Family Health Survey (NHFS) reporting 90% of households having access to mobile phones. However, less than half of women surveyed (46%) have access to mobile phones.40
India has a federal health system structure, where health is a state subject but national government defines key strategies and programmes.41 For example, the Ministry of Health and Family Welfare (MoHFW) is responsible for national programmes for health and family welfare, prevention and control of communicable diseases, promotion of traditional and indigenous systems of medicines, and setting standards and guidelines, which state governments can adapt. Additionally, The Ministry of Women and Child Development (MoWCD) is responsible, among other programmes, for implementing the Integrated Child Development Services (ICDS) programme, in collaboration with the MoHFW, which provides a package of services including supplementary nutrition, immunisation, health check-ups and referral services, and preschool education.
The 2015–2016 NHFS reported that utilisation of key maternal and newborn health services are variable and characterised by breaks in the continuity of care. While most women (83%) attend at least one antenatal care visit, only half (51%) receive the recommended four visits. Despite high skilled birth attendance (81%), provision of postnatal care is uneven for mothers and newborns; with only 65% of mothers and 27% of newborns receiving postnatal care within 2 days of birth.40
Cases
In this study, programmes were considered to be sufficiently scaled up to serve as case studies if they were reaching a large proportion of eligible FLHWs across at least one state in India.
Case studies of varying complexity were selected based on three features. First, our cases showcase a range of technical features such as data capture, decision-support, direct-to-FLHW health information messages. Second, they are at different levels of maturity in terms of scale and sustainability, which enabled us to explore differences in their experiences in scaling and varying levels of success in being sustained. And third, they are geographically diverse, enabling an examination of contrasting Indian governmental state capacities. Our cases and respective digital tools are as follows: (1) TECHO +in Gujarat; (2) Mobile Academy (MA) in Gujarat and Madhya Pradesh; (3) Anmol in Madhya Pradesh and at national level; (4) the non-communicable diseases (NCDs) App at the national level and (5) Common Application Software (CAS) at the national level. Table 1 compares and contrasts each case study in terms of their components and functions, actors involved and coverage.
Case study overview
Data collection
We conducted semistructured in-depth interviews with respondents identified using investigator contacts and snowball sampling in person in New Delhi, Bangalore, Bhopal and Ahmedabad from May to October 2019, and remotely using Zoom software from July to October 2020. Respondents were sampled from the following categories: technology partners, implementers and technical partners (n=11); senior government stakeholders who had played key roles in commissioning, scaling and/sustaining the digital tools (n=5); funders (n=1) and evaluators/academics (n=3) (table 2). Our sample size was limited due to COVID-19 pandemic starting in the middle of our study, which impacted respondents’ availability, as well as our ability to follow up with them in person. However, several respondents (n=7) had in-depth knowledge of multiple cases: MA (n=8); TECHO+ (n=5); ANMOL (n=5); NCD app (n=4); CAS (n=9).
Key informants Interviewed by respondent type with knowledge of specific case studies
Research began by introducing the participant to the study, providing them with an information sheet and consent form for the study, and obtaining and recording verbal consent only after giving them sufficient time to consider whether or not to participate in the research and answering any questions they may have. Interviews were conducted in English using a semistructured interview guide covering domains in the conceptual framework (figure 1). Interviews lasted approximately 1–1.5 hours. At the end of each interview, respondents were asked if they knew anyone with experience relevant to the subject of our research. Researchers took detailed notes and audio recorded each interview, which was transcribed for analysis.
Data analysis
Analysis involved the following stages: NSS and KS systematically coded the interview transcripts using Dedoose software, adopting a framework approach whereby a priori and emerging themes were applied. KS prepared a detailed synthesis report that summarised findings by emerging themes and NSS, KS, AG and AEL participated in an analysis workshop, where emerging findings were reviewed and jointly agreed and the conceptual framework for the study was revised to reflect the study’s findings. Data were then revisited using the revised conceptual framework with the paper drafted by NSS with inputs from KS and reviewed by all authors to confirm the findings are accurately and coherently presented.
We followed Noble and Smith’s recommended steps to enhance the validity and reliability of qualitative data collection and analysis, including accounting for personal biases, frequent communication with all researchers in the study team and ongoing critical reflection of methods to ensure sufficient depth and relevance of data collection and analysis.42 We operationalised these steps through convening planning and debrief meetings before and after each KI to: revisit the interview guide and focus our interview strategy, discuss the detailed interview notes and high level summary comments, and continually re-evaluate our impressions and interpretations of responses to ensure that personal bias was minimised. A preliminary analysis report was also reviewed by the team and discussed at length before proceeding with drafting the manuscript.
Patient and public involvement statement
Given the nature of our study—a high-level policy analysis—it was not appropriate to involve patients or the public in our research.
Results
We now apply the framework to our cases, focusing on the potential impact of salient features within the domains (digital solution, actor roles and relationships, implementation processes and context) on the digital health solution’s scale-up, and the implications of these aspects of scale-up on the sustainability of these solutions going forward.
Digital solution
Here, we report characteristics of the digital solution that affected scale and are likely to affect sustainability: their perceived value to all stakeholders, bounded adaptability and data governance.
Perceived value
Findings suggest that the perceived value of these digital health solutions arose from responding to the needs of various actors. MA was valued as an affordable, standardised and logistically simple mechanism to refresh Accredited Social Health Activist’s knowledge gained from face to face trainings and fill in knowledge gaps. For the range of digital health solutions focused on FLHW data capture, workflow planning and/or decision support (ANMOL, TECHO+, CAS, NCD app), respondents reported valuing the potential to streamline data collection systems and improve the timeliness, quality, accessibility and use of data.
While fulfilling a clear health system need enabled these digital tools to scale, respondents felt that sustainability hinged on tools becoming inherent to effective service delivery. Respondents argued that this underlined the success of one state-level solution (KI14, evaluator/academic), while another speculated that sustainability was not guaranteed because the transformative potential for service delivery was yet to be grasped.
I think to some degree, yeah, sustainability is a pipe dream. … Like, digital is viewed as an IT product, it’s viewed as a data collection mechanism to get you reports. They don't use the reports to do anything useful. So, they're not buying the digital health technology … because it improves outcomes… (KI16, technology partner)
Adaptability
Most key informants identified adaptability as a key enabling feature for scaling up, but one that was bounded by certain technical nuances. They distinguished between software that is hardcoded versus configurable. In order to make changes to hardcoded software a senior engineer is needed to manually rewrite the code; a process which can be time consuming and expensive. By comparison, configurable software allows technology partners to quickly and easily configure different parameters without editing the source code. They specified that configurability was a more scalable approach and seen as essential for accommodating differences in state-level health programmes and priorities. For instance, MA was configurable because new curriculums with different numbers of chapters and quizzes could be easily set up using the existing software and infrastructure (KI01, technical partner). However, informants identified that configurability should be carefully managed and permissions based because allowing states to customise data fields would hamper implementation and impede efforts to aggregate and compare data.
In addition to adaptability, informants noted the importance of extensibility—defined as the ability of systems to extend to and accommodate new users or geographic areas. Extensibility was driven by infrastructure and telecommunications connectivity choices. For example, an interactive voice response system is more extensible if it can quickly and easily scale to handle spikes in call traffic and then descale when the spikes have subsided.
Data storage and governance
Although respondents agreed that data storage and data governance domains (access, privacy and consent) are key considerations for any digital tool, they did not cite them as a key factor contributing to scale or sustainability. Data storage was mentioned as closely tied to scale-up was the use of high-quality cloud-based data storage rather than physical data storage infrastructure. Government resistance to cloud-based systems, which partially stemmed from concern about data being housed outside the country and ‘exposed’ on the cloud (KI04, technological partner), was reported to be rapidly declining as India-based cloud systems have become available and demonstrated their capacity.
Data governance domains including consent, privacy and data protection were not cited as considerations when scaling or sustaining these digital tools. A respondent speculated that some stakeholders viewed data protection as ‘a luxury for wealthy consumers and that it’s a barrier to saving lives, and it’s totally unrealistic for illiterate poor people who can’t, who have no idea what data even is’ (KI01, technology partner). Low prioritisation of data governance domains had resulted in poor consent processes at the front lines, wherein FLHWs were ‘not really’ taking proper informed consent when collecting data using digital tools (KI10, government). Another respondent worried that failure to invest in consent processes could someday undermine sustainability by undermining public trust:
Explaining these [data protection] concepts to poor, illiterate people is super challenging…[but] if you don't engage and take consent to take people’s data, you have a situation of distrust. (KI01, technology partner)
Actor roles and relationships
In this section, we consider the government champions, networks and sustained leadership, and engagement with FLHWs, including the relationships involved in developing, implementing and scaling these digital tools that are likely to affect sustainability.
Government champions
Champions in influential government ministerial positions in MoHFW and MoWCD and at state government level were vital to the scale-up of all digital tools studied (KI14, evaluator/academic). For example, CAS was made possible by having ‘the right guy for the job’ in government (KI06, donor) who was ‘able to push through all the bureaucratic kind of machinery that was required and to make sure that it (CAS) was moving’ (KI17, technology partner).
Communication between state government actors was an important enabler for scaling MA and the NCD app. In these cases, the programme was first implemented in one state with support from senior actors in that state’s government. These actors went on to praise the programme during interactions with their counterparts in other states, which created champions for the programmes in other states.
Stakeholder networks
While a powerful government champion can push a programme forward and approve the decision to scale, sustainability required deeper relationships ‘so it can go all the way down to the people who are actually making things happen’ (KI02, technical partner) and broader buy-in to ensure that the programme continues—especially when individual leaders inevitably move on (KI14, evaluator/academic).
Implementing partners, including non-government organisations, technologists and donors invested heavily in building and sustaining relationships, particularly with bureaucrats at different levels of government (KI14, evaluator/academic). For example, respondents from the technical agency that developed MA described ‘walking the corridors of the National Health Mission’ over several months to build trust (KI02, technical partner).
Beyond government, all our cases showcase the importance of committed, sustained teams and strong leaders at donor, technology, technical, and implementing organisations.
It’s easy to sort of keep looking externally but even internally, if something had happened and our key, you know, drivers of the project internally had left or were involved in something else and therefore not giving this so much time we wouldn't have had the kind of success we had. (KI02, technical partner)
FLHW engagement
While digital tools are meant for FLHWs who ultimately determine their use, several respondents noted that FLHWS were not involved in decision making related to scale-up, beyond participating in implementation and research directed by others. As all the tools were required—not optional—for FLHWs, high uptake cannot be considered a proxy for FLHW endorsement. However, a respondent noted that ‘in the end… the true mark of sustainability [of these digital tools] will be that this is something that gets routinised into the day-to-day work of the health worker themselves so they value it, that they demand it, that if it breaks or crashes or does not work that they want it fixed because they find it so valuable’ (KI14, evaluator/academic).
Implementation processes
In this section, we examine the process underpinning the successful scale-up of these cases, summarised as investments in research evidence, operationalisation considerations and balancing actor roles.
Investing in evidence
Formative work to support fit with context and population
The cases underwent diverse development processes with implications for their fit with the context and population. While some underwent extensive user testing and embedded development (MA, NCD app, TECHO+), others were first developed by technology partners with specifications/data capture features stipulated, and then adjusted when implemented (ANMOL).
For instance, the NCD app arose from an embedded development process, wherein app developers spent 1.5 years working in a primary health centre ‘lab’ with a small group of FLHWs to develop a locally appropriate digital tool (KI03, technical partner). In contrast, some of ANMOL’s implementation challenges were linked to what may have been poor initial fit with the population. A respondent felt that although ‘as an application it’s quite good’ it was ultimately made by ‘someone sitting in Delhi’ in order to generate better data for central decision making and analysis, not for the FLHWs who struggled with using smartphones and lower-level health system managers (KI09, government).
Evidence from research
Both technical partners and government officials suggested that evidence from research studies was less important to informing scale-up than showcasing feasibility (KI01, technical partner) or filling a clear need identified by government officials (KI06, donor).
It’s like, okay, so they [the government] had some evidence… But ultimately, they were interested in the fact that here’s a piece of software that people not only know how to use, but want to use, and are willing to pay for.” (KI01, technical partner)
All our cases were scaled and on track to be sustained, so ones with rigorous research showing impact (MA; precursors to CAS and TECHO+) did not appear to be better placed for long-term sustainability compared with cases with only anecdotal insight from implementation experiences (NCD app, ANMOL).
Nonetheless, respondents involved in CAS suggested that evidence played two positive roles in achieving scale-up. First, although ‘politics were far outweighing the evidence’ policy-makers did need some evidence of effectiveness in order to meet a requirement before going to scale (KI16, technology partner). Second, the process of evaluating a digital tool can improve it by cultivating a ‘learning culture’ (KI14, evaluator) among policy-makers, technology partners and implementors whereby actors sought data from baseline and process evaluation research to improve the programme.
Operationalisation
Rolling out use by FLHWs
While some tools like TECHO+made an effort to address what users at different levels needed and thus reportedly achieved high FLHW satisfaction, several tools initially struggled with FLHW engagement during implementation—however, these initial hiccups did not seem to affect the tools’ eventual scale-up. These challenges included some states failing to distribute advertisements although MA was supposed to be implemented alongside a media campaign explaining the course to FLHWs; and ANMOL being rolled out through trainings for 250–300 Auxiliary Nurse Midwives (ANMs) at a time, which was an unwieldy number of participants (KI05, technology partner/implementor).
Respondents had varied perspectives on whether to rapidly switch from paper registers to digital systems, or to have dual systems for a while before the transition to digital systems is complete. ANMs using ANMOL and TECHO+ are expected to continue maintaining their paper registers. The technological partner working on TECHO+ (KI13) explained that paper record keeping should be phased out gradually, and only after it is clear that the new digital system is working. Moreover, FLHWs are answerable to multiple actors beyond the health system, such as the district collector or the national census, who expect to refer to paper registers.
I completely don’t think it’s their fault because the demands of the health and nutrition programmes of data to be on paper is huge because it’s not just their programme who ask for the data, they have the census guys coming in and say hey show me the register. She can’t say I have everything on the phone I've stopped maintaining a register. (KI06, donor)
In contrast, the nutrition system, ICDS, decided to quickly switch Anganwadi workers (AWWs) from paper to digital record keeping through CAS in order to streamline front-line processes and avoid creating inconsistencies between the two data sources. Removing paper-based records was argued to increase CAS’s sustainability prospects, since it would be difficult to remove a digital system once the paper-based records were gone: ‘You will meet your goal if you get rid of paper registers’ (KI06, donor).
Procurement hurdles
Procurement, whether of handsets, maintenance contracts, server capacity, or platform services, emerged as a major challenge during the implementation process with implications for long-term sustainability. Procurement was not challenging due to financial shortages, but because government actors feared taking major contracting decisions (KI11, technology partner). Government actors are scrutinised when they issue large contracts and can be questioned about its transparency and appropriateness even years after signing off on procurement (KI11, technology partner). Nonetheless, a respondent (KI16, technology partner) also reflected that the converse of this reluctance was the speed of largescale procurement when a government actor identifies an opportunity to ‘get a piece’ of an expensive contract for personal benefit.
Programme developers and funders suggested that sustainability could be fostered by ensuring that government procurement started at the outset, rather than at the time of transition (KI02, technical partner; KI16, technical partner). This would ensure that procurement processes are developed long before handover and avoid creating an expectation that funders or technical partners would handle procurement over the long term.
Evolving stakeholder roles and relationships
Evolving roles in implementation
Once the decision to scale was approved, a tool’s success depended on the implementing organisation’s commitment and capacity in terms of project management, training and roll-out logistics, ongoing testing and improving of the tool and dedicated relationship building across actors including the government.
Despite being used by government, all cases were developed and implemented through extensive non-governmental inputs (table 1). In terms of sustainability, respondents reflected that although the most intense inputs are required early on, ongoing field presence is necessary to troubleshoot, upgrade software and provide sustained support to FLHWs. Planning for this handover to government was not clearly outlined for some digital tools, potentially creating a sustainability challenge. For example, the NCD app implementor (KI11) explained that his organisation did not plan to be involved ‘forever’ but lacked a realistic transition plan agreed by MoHFW. Although the government has developed institutions for managing digital systems and data, for example, National Informatics Centre, respondents diverged in whether they envisioned the government as likely to gain the capacity to effectively manage these solutions over the long-term, in terms of implementation, technological expertise and management, and strategic/technical inputs.
Donor and government funding commitments
Although donor contributions are not significant in terms of overall development funding in India, for all cases, donor commitment was key to scaling up digital tools for FLHWs because there is an ‘expensive curve’ at the beginning which requires external inputs (KI03, technical partner), and because donors can bring innovations in faster than the government, which in turn saves the government time and energy (KI06, donor). Respondents concurred that a digital tool must have a major donor offering to pay for initial phases. In fact, a respondent speculated that many programmes fail to move beyond the pilot phase because they lack sufficient external financial backing to spark government interest (KI03, technical partner).
In terms of sustainability, some respondents suggested an ideal transition arc, with donors taking the initial risk, demonstrating proof of concept, and then having the government take over. Furthermore, a few respondents said that in order to be sustainable, programmes needed to no longer be dependent on donor funding due to shifting donor priorities (KI11, implementor; KI03, technical partner; KI06, donor).
I think donors get bored… they want to go on to the next project. The Programme Officers are worried about their career, they're worried about the optics. […] Don't assume because you're doing good work you can get more [from the donor] … [Donors are] willing to go fund the next random country just because of politics and stars aligned with, you know, what was a priority at the [donor’s], what was the priority for that Minister, and a conversation had happened (KI16, technical partner)
However, other respondents did not consider government funding as a golden bullet to sustaining digital tools for FLHWs, as even with government funding, programmes could still be cancelled (KI01, technical partner). Furthermore, several respondents did not consider sustainability in light of donor versus governmental funding (KI09, government; KI02, technical partner).
Beyond the source of funding, respondents noted examples of programmatic adaptations made to conserve finite resources. For direct-to-FLHW solutions like MA, to limit the risk of unplanned and unaffordable call costs generated by non-FLHWs, the programme used an approach called ‘white-listing’. Only FLHW numbers registered in government tracking databases were allowed to access MA. Any number that called MA that was not registered would hear an audio message indicating that the programme was for FLHWs and if the number had been changed, they should register the new number to receive access to the service.
Context
Data from key informants highlighted national-state dynamics and ministry and programme fragmentation leading to interoperability challenges as influential for scaling and sustaining digital tools.
National-state dynamics
India’s federated structure generates a tension between national- and state-level decision-making, ownership and control. There is no standardised model for national- to state-level scaling up. When the national government decides to scale a digital tool, they offer it to states alongside funding to enable uptake.
This federal structure limited scale-up of the NCD app to a certain extent because states with higher capacity and their own state-level systems (eg, Tamil Nadu, Kerala) were hesitant to switch to the new national government-supported programme (KI11, technology partner). In contrast, TECHO+ was solely developed in Gujarat state with extremely strong state buy-in, but no national profile. A national government actor (KI12) lamented the missed opportunity of expanding TECHO+, explaining that Gujarat created a ‘beautiful’ application but since it only works within Gujarat’s Reproductive Child Health (RCH) health information system, it cannot be taken to other states.
Respondents did not agree on the ideal balance between national authority and state-level control. Some respondents wanted national government to take the lead in selecting tools because state governments ‘are so busy moving from one crisis to another’ (KI09, government) or because nationally standardised tools enable rapid and efficient implementation (KI06, donor). Others wanted the national government to provide leadership on setting standards and definitions for interoperability (eg, semantic interoperability to standardise the names for drugs) but to allow states to build solutions themselves. A senior national MoHFW official (KI12) felt that ‘innovation has to be everywhere’ and not only a national function; furthermore, national government should ‘create an ecosystem’ and standards within which states can innovate. If this standardisation is in place, each state can have its own system using a common application programming interface (API) and all the systems can speak to each other and state level data can be fed into a central database (KI01, technical partner).
Interoperability
The reality of disparate, vertical applications being implemented, often in response to government requests for a single-condition solution, was reported to impede sustainability and contribute to an environment of multiple solutions being implemented in overlapping geographies and across same FLHW cadres. Respondents noted that limited intersectoral collaboration between government ministries and within the health system resulted in the creation of parallel digital platforms that are not interoperable and entrench a siloed approach to the delivery of healthcare, nutrition and other social services. The challenge of interoperability between digital tools within the MoHFW is enormous, but even if they all aligned with the underlying RCH platform, there are multiple other ministries building digital tools that are not interoperable with RCH (KI03, KI04, technology partners).
The most striking example of ministry fragmentation can be found with CAS in the MoWCD and multiple tools in the MoHFW. At the village level, AWWs use CAS, and interact regularly with ANMs, who use ANMOL, TECHO+ and/or the NCD app. Notably, CAS operates on Dimagi’s CommCare platform, which does not integrate with RCH, which is the platform with which the ANM’s digital tools interoperate.
If ICDS-CAS was interoperable then you would be able to integrate the data you have collected from ICDS-CAS with [RCH]. We are talking about probably the same beneficiaries in two different systems […] which is not great (KI04, technology partner)
While the digital tools serve the same communities and same people, particularly pregnant and postpartum women and children, they require FLHWs from two different programmes (nutrition and health) to separately enumerate beneficiaries and collect similar datasets about them. Moreover, malnutrition or health issues identified by FLHWs in one programme and flagged through one digital tool will not integrate on the other tool, thus institutionalising gaps. One respondent (KI14, evaluator) noted that despite this fragmentation arising from the separate government ministries, FLHWs themselves often collaborated, and several states showcased positive collaboration. Respondents who had participated in efforts to integrate CAS and RCH were pessimistic about resolving national-level fragmentation because of ‘egos’ (KI14) and ‘turf protection’ (KI16). They also explained that in the interest of individual departmental needs and requirements, it appeared that both Ministries wanted to subsume the other’s digital system.
Discussion
Given that so few digital programmes scale and then sustain, our study examined several scaled digital tools for FLHWs in India to understand both scale and sustainability of digital health programmes. While digital programmes are assumed to be amenable to scale due to the promise of technological innovation, we found not only technical complexity of digital tools, but also significant systems or relationship complexity, requiring iterative governance approaches to relationship building.
Frameworks for scaling up interventions consider the nature of the intervention and its delivery strategy, attributes of the actors involved and elements of the sociopolitical context.8 Studies of scaling up interventions concur in their conclusions about the unpredictability of the process and the concerted relationship building involved. The technological complexity of digital health programmes further amplifies the importance of such conclusions, particularly because they are not automated solutions, but require significant technological, social service delivery and governance adjustments.13 32 33 36 Not only do they require ongoing support from stakeholders outside the health system (technology partners) whose motivations or incentives may or may not align with health system actors, but they must overcome implementation challenges that are unique to digital tools (interoperability, data governance, ongoing innovation and technological support).43
While factors enabling scale and sustainability did overlap, respondents also noted several nuances that distinguished between them (table 3). Respondents emphatically noted the importance of state-level and national-level policy champions for successfully scaling digital tools; however, sustainable innovations also need long-term sustainable support and buy-in across levels and actors, especially in government. These cases also illustrate the importance of committed teams and strong leaders across donor, technology, technical and implementing organisations. As policy champions are political actors who are often reappointed to new positions or leave government if there is a change in party leadership, to be sustainable, digital tools need strategies to engage a churning succession of leaders and a mechanism to build and sustain relationships with bureaucrats below the policy champions.44 45
Factors influencing scale and sustainability of digital tools for front-line health workers (FLHW) in India
For all cases, we found that genuinely productive partnerships with the potential to sustain digital solutions require working with ministries that ‘put some money on the table’ from the beginning, along with a major donor offering to pay for initial phases (KI06, donor). Once the decision to scale was approved, success depended on the implementing organisation’s commitment and capacity in terms of project management, training and roll-out logistics, ongoing testing and improving of the tool, and dedicated relationship building across actors including the government. Those seeking to develop, scale and sustain digital tools should note the two central themes related to partnership engagement: the importance of investing in relationships and the power of communication between actors within government. These findings are similar to other studies examining national scale-up that found invoking policy champions and building implementer capacity are key ingredients to catalysing scale-up.10 46 47
Planning for handover to government needs to be clearly outlined. Procurement, whether of handsets, maintenance contracts, server capacity or platform services, emerged as a major challenge during the implementation process with implications for long-term sustainability. Even once a tool is successfully scaled-up, the handover to government management can stall, not due to cost but because of procurement, which was challenging not due to funding shortages, but because government actors were afraid of taking major contracting decisions. These challenges are not only faced in more resource-constrained settings, as Lennon et al report similar red tape and ‘procurement pains’ when evaluating a digital health innovation programme in the UK.32
Data highlighted the following components of the government health system context as particularly important for sustaining digital tools: government capacity, dynamics between the national and state levels, and intersectoral collaboration. Siloed approaches to the roll-out of the studied digital solutions leading to interoperability challenges, among others, echo findings from a systematic review of mHealth and telehealth interventions in India that reported most of their included interventions were implemented as standalone solutions, often with no health systems integration strategy.48 It is thus promising to see the recent finalisation of National Digital Health Blueprint (NDHB), which, when implemented, will enforce standards that will allow for seamless interoperability.49 Furthermore, the NDHB will redress data architecture and governance challenges reported in our study and implement some key recommendations made by Tiffin et al50 to reduce the risks of health data breach or misuse in digital health programmes in low-income and middle-income countries, by creating a multitier system that ensures the highest levels of data security and ensuring privacy with Indian citizens as the owners of their data.
Limitations
First, we studied scaled programmes that are in the midst of their implementation, thus our findings, to a certain extent, speculate on what will facilitate or hinder sustainability of these digital tools in the Indian health programming landscape. This means that while the factors that facilitated scale-up (in terms of the digital solution, actor roles and relationships, implementation processes and context) are based on respondent observation of actual practice, their implication on sustainability is speculative and requires longitudinal study. This is a noted limitation in many sustainability studies which do not have longitudinal or extensive retrospective designs.12 15 26 51 52 Second, given our emphasis on understanding policy design decisions rather than implementation experiences, we did not include FLHWs among respondents. Additional research to capture front-line experiences of the digital tool, the scale-up process and the actors involved, as well as their views on sustainability would enrich the story. Third, due to the nature of qualitative research and the limited time available from senior government officials, some components of our framework were not evenly probed in each interview, which may have hindered the comparability of the digital tools studied. Finally, the COVID-19 pandemic impacted the availability of some identified respondents, including additional donors, to be interviewed, which led to a smaller sample size in our study than we had anticipated, potentially reducing the robustness of the case studies.
Conclusion
Digital innovations, despite their potential for scale, are not scaled or sustained in either an automated or linear manner. Donors, tool developers, implementors and government actors seeking to scale and sustain digital tools must focus not only on technical capacity but also on investing in people, relationships and service delivery adjustments to overcome significant challenges and uncertainties. It is imperative that we learn from the complexities involved to realise the potential promises of digital health.
Data availability statement
Data are available on request. Deidentified participant data are available on request from NSS (neha.singh@lshtm.ac.uk).
Ethics statements
Ethics approval
Ethical approval for this study was granted by institutional review boards (IRBs) at Sigma, New Delhi, India, and at the London School of Hygiene and Tropical Medicine, UK.
References
Footnotes
Handling editor Seye Abimbola
Twitter @neha_s_singh, @ashageorge72
Contributors AEL, NSS, AG and RV conceptualised the study and all authors contributed to the final structure and content of the paper. NSS, KS, AG, AEL and RV collaborated on the data collection. NSS and KS led the overall data analysis, and AG, AEL and RV contributed to specific sections. NSS wrote the first draft of the paper with inputs from KS, and all authors contributed to critical interpretation of the results and development of the paper. All authors saw and approved the final version.
Funding We received funding from Countdown 2030 supported by the Bill & Melinda Gates Foundation (award number OPP1148933). AG is supported by the South African Research Chair’s Initiative of the Department of Science and Technology and National Research Foundation of South Africa (grant no 82769) and the South African Medical Research Council.
Disclaimer None of the funders had any role in study design, data collection, data analysis, data interpretation, or writing of the manuscript.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.