Article Text
PDF
Abstract
Seasonal malaria chemoprevention (SMC) is a World Health Organization-recommended intervention to protect children under the age of 5 in Africa’s Sahel region. While SMC remains highly effective in decreasing malaria cases, implementing countries face several challenges regarding collecting quality data; monitoring coverage and compliance and overcoming delays in campaigns due to late payment to field distributors.
To address these challenges, the National Malaria Control Programmes of Benin, The Gambia, Ghana and Nigeria introduced digital data collection (DDC) tools to support their SMC campaigns. To facilitate cross-country learning, this paper investigates the impact of using DDCs in SMC campaigns by comparing country responses.
Country experience suggests that in comparison to paper-based data collection systems, using DDC tools help to overcome data quality and operational challenges; cloud-based features also made data more accessible. Thus, scaling up DDC tools and linking them with routine national health management systems could help generate robust evidence for malaria policy development and programming. Of note, evidence from Benin showed that using digital tools reduced the time to pay staff and volunteers by 5 weeks. In Benin’s experience, DDC also offered cost benefits (1.5 times cheaper) versus the use of paper-based tools.
The authors note that no application offers greater benefits than the other—countries will select a technology that best suits their needs. Several applications are currently being used and newer ones are also being developed. Another option is to develop in-house applications that can be adjusted to local health programmes.
Cost-effectiveness studies to inform on whether DDCs offer cost advantages would be beneficial. More studies on DDC are needed from SMC-implementing countries to identify additional benefits and drawbacks of digital applications. These will similarly help national malaria policy and programming efforts.
- malaria
Data availability statement
All data relevant to the study are included in the article.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Summary box
Country experience suggests using digital data collection tools for seasonal malaria chemoprevention programmes helps to overcome data quality and operational challenges that PBDC tools have. The cloud-based features of digital data collection also made data more accessible.
Evidence from Benin also showed that using digital tools reduced the time to pay staff and volunteers by 5 weeks and was 1.5 times cheaper than using paper forms.
Additional cost-effectiveness analyses are recommended to better assess the full financial benefit of using digital data collection tools.
Scaling up digital data collection tools and linking them with routine healthcare digital data collection systems or National Malaria Data Repositories could help achieve the health-related objectives of the Sustainable Development Goals.
Introduction
General context: use of field DDC applications for health systems
Health information systems (HISs) are one of six vital components of an effective health system.1 An efficient HIS should produce reliable and timely information available for analysis, to identify progress, problems and needs; this will ultimately contribute to the development of evidence-based policies for health programmes.1 HIS also supports the optimal use of scarce resources within health systems, while enabling scale-up of efforts to achieve better health outcomes.2 3
However, weak HIS makes low-income and middle-income countries struggle with challenges arising from the use of paper-based data collection (PBDC) systems: for example, poor data quality, heavy reporting burden on frontline workers, siloed data systems and limited capacity for analysis and use at national and subnational levels.4–6 This leads to adverse consequences for health systems—in the absence of good-quality data, critical health decisions will be based on estimates that may not reflect the real needs of the population.1 Moreover, lack of complete and accurate data hinders performance assessments. Thus, suboptimal HIS can be a barrier to achieving the health-related objectives of the Sustainable Development Goals and attaining Universal Health Coverage.7
To strengthen HIS, some countries have introduced digital data collection (DDC) tools to support field data collection and management for a wide range of health interventions. In the case of malaria, mobile DDC tools have been used to collect data on malaria case management, vector management, clinical trials, national surveillance and household surveys.4 8–10 Typical tools include the Malaria Control System and personal digital assistants to collect survey data.4 8–10 Apart from making data collection more robust and reliable, these tools have several benefits, such as making data collection quick and easy and improving quality by minimising errors. They also allow for near real-time data visualisation—which supports monitoring, evaluation and further analysis for decision-making.4 9 10
Specific context: use of DDC tools for seasonal malaria Chemoprevention
In 2020, an estimated 627 000 lives were lost to malaria globally.11 Of these, 96% lived in Africa and 80% were children under the age of 5.11 To protect children from malaria, in 2012, the WHO recommended seasonal malaria chemoprevention (SMC) in areas of highly seasonal malaria transmission during the rainy season, during periods when malaria incidence is at the highest.12 Pooled estimates across seven studies in SSA indicate that SMC implementation reduces the incidence of clinical malaria during the first 4 weeks by about 88%.13 Since 2012, 13 countries in the Sahel region of Africa have implemented SMC and over 33 million children were protected by the intervention in 2020 alone.14
For SMC, caregivers administer three doses of the drug, sulfadoxine–pyrimethamine plus amodiaquine, over a 3-day period for up to 5 months of the rainy season.12 Children eligible to receive SMC include those aged between 3 and 59 months at the time of the first monthly treatment of the year who were not unwell, had no known allergies to SMC drugs and had not taken amodiaquine (AQ), sulfadoxine–pyrimethamine (SP) or sulfa-containing antibiotics in the previous 4 weeks.15
In each month of the SMC campaign season, also known as a cycle, the caregiver gives SP only on the first day by directly observed therapy. Subsequently, the caregiver or sometimes volunteers/health workers during home visits administer only AQ on days 2 and 3.15 This is done each month for up to five cycles, to protect eligible children from malaria—this constitutes one SMC round.16 The commencement of an SMC round depends on the start of the rainy season.15 In the future, weather information to trigger the start of an SMC campaign could well be digitally obtained.17 Of note, new COVID-19 guidelines direct mainly caregivers to administer doses (rather than health volunteers or staff) to minimise transmission.18
Challenges in SMC implementation
In implementing SMC, countries noted that beyond the challenges related to quality data collection, several other factors limited its successful rollout. First, SMC is a multidose intervention that occurs over several months, thus caregiver compliance with all the doses and cycles remain challenging.14 Second, countries also report delays in starting cycles—this is associated with timely payment to health volunteers and correct assessment of their work. Delays in this crucial first step impedes administration of the second and third doses.14
Third, some countries combine SMC with other interventions, such as the distribution of insecticide-treated bed nets (ITNs), screening for tuberculosis and malnutrition and malaria case management.14 While this is a good way of optimising scarce resources, it places an additional data collection workload on field teams.14
Recently, countries such as Benin and The Gambia piloted the use of DDC applications in ITN campaigns to address data quality concerns and to efficiently monitor coverage rates.19 20 Based on the lessons learnt from this pilot in their own or other countries, four SMC-implementing countries—Benin, The Gambia, Ghana and Nigeria—decided to apply DDC tools to overcome the data challenges within SMC campaigns. This study presents their experiences, lessons learnt and recommends next steps.
Overall approach
A panel discussion was organised at the virtual March 2021 SMC Alliance21 meeting. The event hosted ~200 participants from National Malaria Control Programmes (NMCPs), SMC-implementing partners, partners working in child and maternal health and representatives from the national and international malaria donor community. Three NMCP representatives (one each from Benin, Ghana and Nigeria) and two representatives from the Catholic Relief Services (CRS) in The Gambia and Benin, shared their experiences of the introduction of android-based DDC tools during countries’ 2020 SMC campaigns.
The representatives covered several topics including rationale for introducing DDC tools; and how they were introduced, used to register children and monitor compliance, coverage and pharmacovigilance. They also provided information on the extent to which DDC tools facilitated the payment of distributors and supervisors, and whether data were collected for interventions other than SMC. Finally, they shared their views on whether DDCs were more cost-effective than PBDCs; they closed by presenting any future plans for scaling up digital applications.
A thematic comparative analysis (table 1) was conducted to facilitate assessment of their responses, and a descriptive design was used to present the benefits and lessons learnt from the incorporation of DDC tools in SMC campaign implementation.
Summary of results from using digital data application tools for implementing SMC
Comparing different DDC tools
Benefits and lessons learnt from using DDC tools for SMC campaigns
We present the responses from each country, grouped according to the various themes investigated in this analysis.
Benin
Rationale for piloting DDC application
To improve data quality, reduce workload on workers and ease management of and accessibility to data.
Application deployed
Benin deployed Red Rose, an android-based mobile technology application, which was designed using the cash and asset technology strategy developed by CRS.22 It was used on mobile phones in four health districts in Northern Benin. Typically programmed to receive biological information in real time, the application also provides proximity and navigation information for eligible children living in households within reach of field distributors.
Registration, dose compliance and coverage monitoring
An existing population data set, obtained from a microcensus enumeration exercise conducted as part of the country’s ITN campaign, was used to implement the SMC campaign, saving costs for the programme. From this existing data set, data for children under the age of 5 were extracted. Children whose details were not in the data set were registered.
To enable drug administration, each child was given a badge with a unique quick response code to keep at home. Based on the information provided at household visits or from within the database, healthcare workers (HCWs) determined which children were eligible to receive SMC and administered the first dose to them. In areas where caregivers administered the second and third daily doses, the child’s barcode was subsequently used to monitor dose compliance for each phase of the SMC campaign. To promote dose compliance, caregivers received reminder SMSs to administer subsequent doses.
Data from Red Rose were linked with Aeronautical Reconnaissance Coverage Geographic Information System to develop near real-time dose coverage maps, enabling supervisors to track the therapeutic and geographical coverage rates. The application was also used to monitor adverse drug reactions and the children’s general state of health.
Health worker performance assessment
According to the team in Benin, when digital tools were used, staff and volunteers were paid in 3 weeks compared with 8 weeks when using paper-based tools.
Faster payment was possible because of a monitoring and supervision tool associated with the Red Rose application which monitored training, attendance and work performance of community HCWs, volunteers and their supervisors. This tool logged the level of work done on the Red Rose app, thus facilitating performance-based payment and detecting fraud.
Using the application for other interventions beyond SMC
In Benin, SMC is currently implemented as a single initiative. However, the NMCP is in discussion with the country’s Primary Health Care Agency (l'Agence nationale des soins de santé primaires), exploring the possibility of combining SMC with malnutrition screening and/or systematic deworming programmes.
Digital infrastructure management
Currently, the application is only on smartphones acquired by the NMCP specifically for the SMC campaign.
To reduce implementation costs, the monitoring and supervision tool was downloaded on the personal devices of volunteers and health workers instead of buying smartphones for all.
Comparing costs
According to the Benin team, the paper-based version was 1.5 times more expensive than the digital system (In West African francs: 567 787 126 CFA estimated cost of the former for two Health Zones in 2019 vs 378 432 033 CFA for the latter).23
Challenges encountered
To address electricity challenges, solar power banks were used to charge mobile devices in areas where electricity was unavailable. The unavailability of internet networks in all localities has led to the use of networks from neighbouring countries.
Future plans
Benin did not state any specific future plans.
The Gambia
Rationale for piloting DDC application
To accelerate complete data collection, data comparison and analysis.
Application deployed
The Gambia first introduced the Apple-based iFormBuilder tool for SMC in 2015, following its experience with it during the ITN mass distribution campaign of 2014. In 2018, the country switched to CommCare, an android-based application, as it was easier to use. CommCare collects data in real-time and has visualisation features that support monitoring and programme improvement.
Registration, dose compliance and coverage monitoring
For each child, caregivers were given a card with a barcode, which was registered on the CommCare application. CommCare was also used to communicate with caregivers on the child’s state of health, for discussion during follow-up visits. Using the application, volunteers were able to capture comprehensive information on a child’s health, including intake of other medications, which was required to determine the child’s eligibility for SMC.
Health worker performance assessment
Payment of volunteers was not based on data generated on CommCare because poor internet connection does not allow immediate data synchronisation after the work is completed.
Using the application for other interventions beyond SMC
The Gambia was the first country to not only digitise its nationwide SMC campaign but to also show that partial malaria campaign integration is possible by sharing mobile devices across SMC and ITN campaigns from 1 year to the next.
Digital infrastructure management
The NMCP purchased mobile devices for the SMC campaigns. Most of the devices have been used for over 5 years and are still working well. To mitigate against loss, a tracking application was installed on each device. Software that blocked the use of other applications was also installed to avoid interference with CommCare’s data collection for SMC.
Comparing costs
Although The Gambia did not mention that they had conducted a cost-effectiveness analysis, they noted that using DDC was cheaper in the long term than PBDC.
Challenges encountered
Information could not be synched daily due to internet connectivity issues. To address technological challenges, a data quality and IT team were put together. In July 2021, CRS in collaboration with the NMCP, trained 78 data collectors and supervisors on District Health Information Software 2 (DHIS 2) tracker,24 were piloted in Kombo South District for the 2021 SMC campaign.
Future plans
The Gambia will explore a move from CommCare to DHIS 2 tracker, which allows for more comprehensive health data collection and supports better monitoring and evaluation.
Ghana
Rationale for piloting DDC application
To improve data quality; facilitate data collation, analysis and accessibility and reduce the workload on community HCWs and distributors.
Application deployed
The SMC team in Ghana introduced the SiCapp android-based application in 2018 for data collection. The application has been rolled out in all SMC implementation areas in Northern Ghana. SiCapp also supports the enumeration of households and children.
SiCapp is customised to the NMCP’s needs. As it was designed by in-house IT staff of the NMCP, it can be easily modified to bridge any gaps identified during programme implementation. This removes the time required to liaise with external developers, which can cause delays during SMC implementation.
Registration, dose compliance and coverage monitoring
Using SiCapp, volunteers registered households within specific communities. They then informally interviewed caregivers and, based on the information received, registered all children within the household eligible for SMC. The first dose of SMC was subsequently administered, and the corresponding data entered into the app. These data were subsequently recalled before entering information on the second and third doses. This helped to identify children who were absent for the first dose—they were then given their first dose during the following dosing rounds. Coverage of SMC was also monitored using the application.
Community health worker performance assessment
SiCapp supports the monitoring of volunteer performance by logging in details of completed work.
Using the application for other interventions beyond SMC
The development of SiCapp for SMC campaigns was inspired by the successful deployment of NetApp for mass ITN campaigns. Both of these applications are android based, but not linked in anyway. This separation allows the NMCP to maintain data integrity. NetApp is used during national ITN campaigns that take place every 3 years, while SiCapp is used every year for SMC campaigns that only take place in targeted districts.
Digital infrastructure management
To save on the cost of buying tablets and mobile phones for all community HCWs, a limited number of tablets equipped with SiCapp were provided by the NMCP. SMC was carried out in a small number of areas at a time and the tablets were then transported to other areas for use. Developing SiCapp in-house also reduced the cost of software development.
Comparing costs
Ghana is yet to conduct a robust cost-benefit analysis. However, the NMCP believes that digitisation is ultimately cheaper in the long term, although the initial outlay may be greater.
Challenges encountered
Like Benin, Ghana experienced internet connectivity challenges in some areas. The team has, therefore, developed and is using an offline version of SiCapp in these areas.
Future plans
Ghana did not indicate any future plans.
Nigeria
Rationale for piloting DDC application
To improve data quality, reduce workload on workers, facilitate payments and ease management of and accessibility to the data.
Application deployed
Nigeria piloted the android-based Reveal application, first introduced in 2020 through a pilot project. The application can identify structures within 5 m of the device and help enumerate children within these households/structures.
From 2021, the NMCP team will pilot Red Rose, an app developed by CRS and used by Benin. This app has been previously used by CRS and partners in Global Fund-supported Nigerian States for the roll out of ITNs.
Registration, dose compliance and coverage monitoring
Using the Reveal app, households and children were registered, and their SMC eligibility determined. A colour-coding feature of the app helps to highlight households that are ineligible, those that have been visited and those where SMC was fully, partially or not administered.
Community health worker performance assessment
The Reveal app can track the work done by personnel and can also provide biometric proof of attendance at training sessions. It can, therefore, provide evidence to support the timely payment of personnel (like existing modules in Red Rose).
Using the application for other interventions beyond SMC
Nigeria reported that the Reveal app had been previously used for ITN campaigns. However, for this pilot, they did not combine SMC with other interventions.
Comparing costs
Nigeria’s NMCP representative said that digitisation is ultimately cheaper in the long term. Like the other three countries, Nigeria had not conducted a cost-benefit analysis to provide evidence for this point.
Digital infrastructure management
To reduce the cost of implementation, the Reveal app was downloaded on the personal devices of volunteers and health workers for use rather than buying mobile phones and tablets for all. This was continued during the pilot of the Red Rose app and will be continued during scale-up as well, given the large number of volunteers implementing SMC.
Challenges encountered
During the app’s first rollout, the team encountered challenges with the need to train community health volunteers with low literacy levels; poor internet connectivity in some areas that affected the syncing of data; and an increase in the number of days required to implement SMC, as it took time for staff to get used to the new system and implement it. Moreover, some of the areas could not be located on the Reveal app. These will be considered for inclusion during the pilot of the Red-Rose app.
Future plans
The NMCP plans to address the challenges confronted in the first rollout; decide between scaling up of either the Reveal or Red Rose application and then link the selected technology with the National Malaria Data Repository.
Comparing different digital tools
To enable countries better understand each digital application, we compared the different tools based on information from the product websites and from the real-life experience provided by the four NMCPs on ease of use, ease of training and disadvantages. This could facilitate decisions of NMCPs wishing to trial the use of DDC for SMC and other malaria programmes.
Other issues cited by respondents across the board for all technologies include the fact that staff can still manipulate the data on the applications and hence initiate fraud. It is also possible to enter erroneous data on the applications, although this is still much less in comparison to PBDCs as the need for multiple transcripts for data analysis is removed. A final challenge is the training time lag in the ability to master the application due to the relatively low level of education of field staff.
Discussion
The use of digital versus paper data collection tools is currently much debated in health circles. Pilot studies on the use of DDC tools in ITN campaigns conducted by Benin, The Gambia and Nigeria sought to understand and address data quality concerns as well as coverage rates of this national ITN distribution programme.19 20 Applying the lessons learnt from these studies to their national SMC campaigns has revealed the advantages and shortfalls of four different DDC applications vis-à-vis PBDC tools. This could facilitate cross-country learning and help NMCPs make informed decisions on the appropriate DDC tool to introduce into their SMC campaign.
Benefits of DDCs over paper-based applications
This assessment finds that, like other studies,3 5 25 DDC tools have numerous advantages in the areas of data collection, quality, monitoring and accessibility for SMC-implementing countries, as is the case in countries implementing ITN campaigns and other health inventions.19 20
This approach is superior to PBDC systems and helps to overcome data quality and operational challenges. Although the possibility of introducing errors via digital tools still exists, it is minimised, as multiple transcripts are not needed. DDC also reduces the bulk use of paper. For this study, digital apps also helped with tracking children and compliance with SMC. This is also supported by Thriemer et al, who report that DDC helped to track patients in their study on replacing PBDC forms with electronic data entry in the field in Zanzibar.26
Visualisation features help monitor coverage while barcode features help monitor compliance. The cloud-based feature made data more accessible. The ability of DDCs to reduce errors and increase data quality, point to their usefulness in generating robust evidence to enable countries to develop effective health policies and programmes, as suggested by Ndovlu et al.27
According to the NMCP teams in Benin and Nigeria, PBDC tools were 1.5 times more expensive than digital tools. Although The Gambia, Ghana and Nigeria did not present a cost-benefit analysis, they all agree that, in the long term, DDC will likely be cheaper than paper-based tools. Taber et al for instance support this point and note that although the fixed costs for DDC were higher, the variable costs were much lower than for PBDC, resulting in a 7.73% lower overall cost.28 Njuguna et al also indicate in their study on the cost-effectiveness of digital-based applications that during their establishment, the cost of DDCs was 9.4% higher than that of PPDC. However, after 2 years, DDC costs decreased by 7%, compared with PPDC. Additional cost-effectiveness analysis will be helpful to quantify the financial benefit of using DDCs; further work in this area is required.
Digital tools also reduced the time required to pay staff and volunteers. In Benin, for instance, staff were paid 5 weeks earlier (in 3 weeks instead of in 8 weeks), courtesy of digital tools.
Benefits of combining SMC with other health campaigns
Some countries did not support combining SMC with interventions, as that would be resource intensive and compromise the quality of coverage due to limited staff. They agreed, however, that gains could be made from using enumeration data (microcensus and/or health data) from other campaigns or other health interventions and vice versa—for example, enumeration data from SMC campaigns for other health campaigns. It would avoid collecting the same information multiple times.
Choice of DDC tools
Based on the comparative assessment of the various technologies, it appears that no technology offers greater benefits than another and each has downsides that countries must consider. For instance, although Red Rose has most of the features, NMCPs require to implement and monitor SMC campaigns (for registration, determining eligibility, monitoring coverage and compliance and assessing the performance of health volunteers and staff), the Benin team commented that it cannot be used to collect additional health data on children, which is a drawback. What is more, although it offers the advantage of running off solar energy and can be used in areas where electricity supply is challenging, the batteries need to be constantly recharged.
Nigeria is also currently testing the Red Rose app and will compare it with the Reveal app. The results and experience from Nigeria will provide useful information to other countries on the additional benefits and drawbacks of these two applications. Meanwhile, the Gambia is currently exploring a switch from CommCare to DHIS Tracker 2.
It was noted that with exception of SiCapp, all other applications are not easy to modify in-house.
Countries that prefer to adjust their local programming needs to their chosen application may opt to follow Ghana’s NMCP and develop an in-house technology. This could be a cheaper option and help avoid constant and costly exchanges with offshore teams when application glitches arise. It would, however require a dedicated in-house software team to manage its development, subsequent roll out and management.
To ensure that the selected application is appropriate, each country needs to assess their needs and match them against the specific offerings each application provides. Countries have several DDC options, and others are continuously being developed. New DDC tools should ensure that the technologies are simple and easy to use, as highlighted by The Gambia, which switched from one digital application to another due to its ease of use and additional functionality. Labrique et al also cited simplicity and value-add as drivers of success for digital health technologies.29
In addition, solutions to reducing the limitations of internet connectivity and power infrastructure, that have also been cited by other mobile health projects, would benefit the digitisation of SMC programmes.30
Conclusion
Our assessment indicates that malaria-endemic countries, especially those in sub-Saharan Africa, continue to encounter data collection and data management challenges when implementing malaria programmes. Given the documented benefits of using DDC systems over PBDC systems, promoting their use, scaling them up and linking them with routine healthcare DDC systems or National Malaria Data Repositories could help achieve the health-related objectives of the SDGs. We recommend further cost-effectiveness analyses as these will help to better assess the financial benefit of using DDCs. Investment in stronger internet connectivity and power infrastructure is vital for SMC-implementing countries to gain the full benefits of DDC systems. We recommend more work in this area to add to the current body of knowledge, as the future is digital.
Data availability statement
All data relevant to the study are included in the article.
Ethics statements
Patient consent for publication
Footnotes
Handling editor Seye Abimbola
Collaborators Jaya Banerji, Consultant, Medicines for Malaria Venture.
Contributors AP-A and A-MT conceived the study. Contributions of experiences, viewpoints and extensive review of the document were done by all other authors. AP-A analysed the information provided, drafted the initial manuscript and finalised the manuscript. All authors were responsible for subsequent revisions and approved the final submitted version. All authors had full access to all the data in the study and had final responsibility for the decision to submit for publication.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Disclaimer The views of the authors are their own and do not necessarily reflect that of the organisations they work for.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.