Article Text
PDF
Abstract
Introduction This paper maps the evidence published between 2000 and 2018 on the use of mobile technologies to train community health workers (CHWs) in low- and middle-income countries (LMICs) across nine areas of global healthcare, including the neglected areas of disability and mental health.
Methods We used an evidence mapping methodology, based on systematic review guidelines, to systematically and transparently assess the available evidence-base. We searched eight scientific databases and 54 grey literature sources, developed explicit inclusion criteria, and coded all included studies at full text for key variables. The included evidence-base was visualised and made accessible through heat mapping and the development of an online interactive evidence interface.
Results The systematic search for evidence identified a total of 2530 citations of which 88 met the full inclusion criteria. Results illustrate overall gaps and clusters of evidence. While the evidence map shows a positive shift away from information dissemination towards approaches that use more interactive learner-centred pedagogies, including supervision and peer learning, this was not seen across all areas of global health. Areas of neglect remain; no studies of trauma, disability, nutrition or mental health that use information dissemination, peer learning or supervision for training CHWs in LMICs were found.
Conclusion The evidence map shows significant gaps in the use of mobile technologies for training, particularly in the currently neglected areas of global health. Significant work will be needed to improve the evidence-base, including assessing the quality of mobile-based training programmes.
- systematic review
- health education and promotion
- public health
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Summary box
What is already known?
The evidence-base on community health worker (CHW) programmes is too weak to recommend specific education approaches or supervision strategies as many programmes are not underpinned by appropriate theories of learning
What are the new findings?
The evidence map shows clear gaps on the use of mobile technologies to train CHWs in low- and middle-income countries across nine areas of global healthcare.
The evidence-base is skewed towards a cluster of evidence on maternal health and child health and conditions with a high global burden of disease (eg, disability, mental health) show little or no coverage.
What do the new findings imply?
There is an insufficient evidence-base to inform policy or practice as 53% of the areas covered by the evidence map only have a single or no study at all.
Social learning has potential for further integration into CHWs training programmes.
Introduction
TheWHO has estimated that at least half of the world’s population cannot obtain essential health services due to a global shortage of health workers.1 One solution to help address this gap has been to train community health workers (CHWs) in low- and middle-income countries (LMICs). CHWs are lay people working within their own community in a health promotion and prevention role.2 As evidenced by major systematic reviews,3 4 CHWs play a crucial role in healthcare for vulnerable populations, resulting in major projects being funded by international donors (Department for International Development, United States Agency for International Development, etc) across Africa and Asia.
Aligned with this, smartphones are increasingly commonplace on sub-Saharan Africa. The Global System for Mobile Communications Assocation, (GSMA) estimates that by 2025 mobile internet penetration will rise to 40% (up from 21% in 2017) using 3G, 4G and 5G (62%, 29% and 3% of connexions).5 It is therefore not surprising that global health researchers, practitioners and policymakers have sought to use mobile phones to increase the reach of their programmes. While there are many mHealth platforms for clinical decision support, very few include a CHW training component in their software. Consequently, the role of mobile technology in the training of CHWs in LMICs remains poorly understood. The methods by which mobile phones can support the different aspects of training and supervision have not yet been fully established.6 In previous work, we found that the design of many training programmes is not underpinning by learning theory.7 These findings are further supported by a 2018 WHO report,8 9 in which the evidence-base for supportive supervision was found to be severely lacking. There remains a pressing need for a comprehensive study of the exact nature of the evidence-base regarding the use of mobile technology to support the training of CHWs. In this paper, we develop an evidence map to address this issue. It details the range of pedagogical approaches and technologies employed for training, and the type of implementations and study designs used across nine areas of global healthcare in LMICs, including the neglected areas of disability and mental health.
Evidence mapping is an evidence synthesis methodology to systematically source and organise a body of knowledge to provide a high-level overview of the size and nature of the available evidence to inform and facilitate the use of this evidence-base.10 Building an evidence map requires ‘a systematic search of a broad field to identify gaps in knowledge and/or future research needs that presents results in a user-friendly format, often a visual figure or graph, or a searchable database’.11 When visualised on an interactive evidence interface (eg, https://africacentreforevidence.org/wp-content/uploads/2018/10/mhealth-and-training-for-chws_v5.html), evidence maps allow decision-makers to directly engage with the evidence and to interrogate its relevance to their own contexts and needs. In summary, the evidence map will answer the following question: What is the role of mobile technology in the training of community health workers in LMICs?
Methods
We conducted an evidence map of existing research evidence investigating the use of mobile technologies to support the training of CHWs in LMICs. Evidence mapping is a research methodology that is part of the family of methods for research synthesis. It thus follows research steps and guidelines for systematic reviews12 and shares systematic reviews core principles of taking a transparent and systematic approach to collecting, organising, appraising and synthesising a body of knowledge.
Detailed methodological introductions to the evidence mapping process have been developed.11 13 14 In short, evidence maps aim to present a systematic overview of a body of knowledge rather than a full synthesis of this knowledge. Evidence maps are therefore broader in scope than systematic reviews and contribute a descriptive overview of the existing evidence-base. Their key contribution is to illustrate and better understand patterns in the existing evidence-base: in particular, the size of this evidence-base (eg, gaps and clusters of research evidence) and its nature (eg, characteristics of the research evidence and systematic patterns in it). While evidence mapping as a research method refers to the key steps of systematic reviews—that is defining inclusion criteria, searching and screening for evidence, data extraction, critical appraisal, summary/synthesis of evidence—the term ‘evidence map’ refers to the output of this mapping process. Such an evidence map usually refers to a to a figure, table or online database/interface explicitly called an ‘evidence map’.11 In our research, the term ‘evidence map’ refers to the visualisation of the collected and organised evidence-base on the use of mobile technologies to support the training of CHWs in LMIC (see Visualisation of the evidence-base section). We next outline the research steps followed in the conduct of our evidence map.
Inclusion criteria
We defined a priori explicit inclusion criteria to determine the scope of the evidence map and what type of research evidence is eligible for inclusion in the map. We used the People, Intervention, Comparator, Outcome framework as a basic structure to organise these criteria for inclusion of evidence and to set boundaries around what to include and to exclude from the evidence map. In order to be included, a study had to meet all of the below criteria. Our general approach was to develop deliberate broad inclusion criteria and to set a wide scope for the evidence map to reduce the risk of missing any relevant evidence. The primary focus of the evidence map is to build a broad evidence-base in relation to our review question rather than to answer narrow questions of what works (how and why, for whom, etc.).
Population
We only included evidence that focuses on CHWs as a target population. However, since the terminology to describe CHWs differs across contexts as do the healthcare practice tasks and functions performed by CHWs, we adopted a broad definition of CHWs defined as ‘members of the communities where they work, should be selected by the communities, should be answerable to the communities for their activities, should be supported by the health system but not necessarily a part of its organisation, and have shorter training than professional workers’.2 This definition allows for different types of healthcare workers to be classified as CHWs in different contexts. In addition, we further included studies that referred to wider primary healthcare professionals working in the community or with CHWs, such as nurses and midwives. This is in an effort to also capture evidence on the role of mobile technologies in the training of healthcare workers, which is likely to be transferable to CHWs. In order to be included, studies did not have to focus solely on CHWs but where a mix of healthcare professional was involved in the intervention, they had to present disaggregated data on CHWs.
In terms of region, we only included evidence that has been conducted in LMICs using the World Bank classification of countries (https://datahelpdesk.worldbank.org/knowledgebase/articles/906519-world-bank-country-and-lending-groups). Where studies included data from both LMICs and HICs, we included these as long as the data were disaggregated for LMICs. Finally, we only included evidence published from 2000 onwards. The year 2000 as a cut-off date was chosen as mobile technologies did not see widespread application to support healthcare in LMICs before then.
Intervention
We included evidence that focused only on the use of mobile technologies in the training of CHWs. Mobile technologies refer to devices such as phones and tablets, but not to fixed technologies such as laptops and desktop computers. We interpreted the concept of training broadly to include any types of activities that aim to facilitate CHWs’ learning practices. This included not only pedagogically grounded activities, such as mobile learning15 16 but also extended to the workplace-based learning17 activities involved in decision-support, provider–provider communication, collaboration, provider work planning and scheduling, and data collection and reporting,18 and definitions of each term can be found in the associated papers.15–17
Study design
We included evidence based on any type of research methodology and design. That is, any evidence generated by an empirical research process featuring a structured process for the collection of empirical data and its subsequent analysis was included. This excluded lab experiments, for example, testing the technical functionality of a technology, where the intervention is not applied in a real-world setting. It also excluded theoretical and conceptual studies such as commentaries, frameworks and opinion pieces. We did not limit the inclusion of studies to particular types of evidence. Both academic literature published in peer-reviewed journals and any form of grey literature (eg, non-governmental organisations (NGO) reports, practitioner evidence) were eligible for inclusion.
Outcomes/phenomenon
In order to be included, studies had to use mobile technologies in CHW training related to either health conditions or healthcare tasks. That is, we did not limit the inclusion of evidence to only specific health conditions (eg, HIV/AIDS) or only prescribed healthcare tasks (eg, screening). However, studies that only evaluated the general applicability of mobile technologies in the training of lay public service staff were excluded.
Searching and screening for evidence
Systematic search for evidence
We developed an explicit and exhaustive search strategy for academic and grey literature evidence to be included in the map (online supplementary appendix 1). This strategy was run in two parts, and when combined covered a total of 62 search sources. In the first part, the search for academic evidence, we developed a Master Search Strings that combined core search concepts (and associated terms) using Boolean operators. The full search string is provided in online supplementary appendix 1 with an example of it provided in box 1. This Master search string was then adapted and applied in eight scientific databases (Pubmed, Medline, Web of Science, ERIC, Education Full texts, CINAHL, Cochrane, Global Health database). We searched on title, keywords and abstract in the respective databases.
Supplemental material
Sample search string
Search concept and Boolean links Community health care workers AND LMICs AND mobile technology AND training.
For example (‘Community health care worker*’ OR CHW OR ‘lay health worker*) AND (‘low income countr*’ OR ‘middle income countr*) AND (‘mobile phone*’ OR tabet*) AND (training OR education)
In the second part, the search for grey literature, we searched the websites and databases of 54 organisations working in the space of mHealth, LMICs and CHWs (online supplementary appendix 1). These organisations ranged from NGOs and inter-governmental organisations to development agencies. Depending on the sophistication of the organisation’s research repository, we either adapted the master search string or engaged in a manual search of the website. In addition, we also directly contacted key organisations and individuals for relevant evidence.
Screening of evidence
All results from the academic database searches were inputted into EPPI-Reviewer software to facilitate screening and data management. We screened the citations gained from the search against our inclusion criteria (presented in Inclusion criteria section) in two stages. In the first stage, all citations were screened on title and abstract. This process was conducted by two reviewers, who screened a subset of 10% of the citations to reach consistency. A third reviewer supported screening decisions through joint discussions where the two initial reviewers did not agree. Following this, the full texts of studies were obtained and screened, again using a similar process of reaching consistency and resolving disagreement between reviewers. Results for the grey literature search were initially screened on title and abstract online; only the studies included for full-text screening where then inputted into EPPI-Reviewer.
Data extraction
Data extraction for evidence maps focuses on broader, higher-level variables than data extraction for full systematic reviews. We developed an a priori data extraction tool in EPPI-Reviewer that extracted the following variables from each primary study: type of training/education; type of health condition; type of CHWs; region; type of programme implementation; type of technology and study design. Two reviewers extracted data from the included studies again following a similar process for reaching consistency and resolving disagreement between reviewers. All data extraction took place in EPPI-Reviewer which was used for data management and reporting. We did not conduct a critical appraisal of the included evidence as the main objective of the research is to provide a high-level overview of the size and nature of the existing evidence-base, rather than to make judgement regarding its methodological trustworthiness.
Visualisation of the evidence-base
We developed a matrix framework to map the included evidence-base on the use of mobile technologies to support the training of CHWs in LMICs. This matrix consists of two variables, against which the evidence is mapped: (i) the type of training/education and (2) the targeted health conditions. We followed an iterative process to develop the matrix framework including a hybrid stakeholder engagement event with organisations who are part of our evidence network (list removed for review).
The matrix columns structured around health conditions were informed by a range of existing health policy frameworks including the Global Burden of Disease report,19 International Classification of Disease (ICD-11) and the Lancet’s report on healthcare funding in LMICs ‘Development assistance for health: past trends, associations, and the future of international financial flows for health’.20 We developed an initial mapping framework based on a synthesis of frameworks included in these reports and on our stakeholder event. Following this, we further iterated on the mapping framework using an inductive approach based on the data extracted from the included studies in the map. In addition, we followed a more deductive approach to the inclusion of more neglected categories of health conditions such as trauma, nutrition, mental health and disabilities to safeguard against not overlooking existing evidence gaps against these categories.
The development of the matrix rows structured around the type of training/education followed a similar process. Here, our starting framework referred to the mHealth and Information and Communication Technology (ICT) Framework for RMNCH.18 From this framework, we extracted the mHealth applications categories relevant to training and education (see our previous work for a more in-depth discussion of this6 7) and then developed these iteratively based on the included studies. Having developed the matrix framework, we mapped the included evidence against it using two different visualisation approaches. First, we developed a heatmap in MS Excel to illustrate the overall gaps and clusters of evidence (figure 2). Second, we developed an interactive evidence map (the interactive interface is available here: https://africacentreforevidence.org/project-outputs-6/) which illustrated the same data (ie, coverage and gaps) but, in addition, allows users to filter the mapped evidence-base against a range of key variables to better understand the nature of the included evidence.
Heatmap of the evidence-base.
Patient and public involvement
Patients or the public were not involved in this research.
Results
Search results
Our systematic search for evidence identified a total of 2530 citations (figure 1). The breakdown of search hits per search sources and a detailed search strategy is provided in online supplementary appendix 2. Grey literature and academic sources contributed a roughly equal number of citations. Following screening on abstract and title, we excluded a majority of citations as not relevant to the scope of the evidence map (n=2079). Full texts of the remaining citations were then sought and screened. This process led to the exclusion of an additional 363 studies with the main reasons for exclusion being that a study did not apply a relevant study design (n=157), did not focus on CHWs (n=81) or did not use mobile technologies (n=77). We included a total of 88 studies (online supplementary appendix 3).
Supplemental material
Supplemental material
Overview of search results and inclusion of studies.
Characteristics of the included studies
The 88 included studies constitute the available evidence-base on the use of technology to support the training of CHWs in LMICs. An overview of all data extracted per study and used for the map is provided in online supplementary appendix 4. A large majority of studies were conducted in LMICs (n=40) and low-income countries (n=37). Only 12 studies were conducted in upper-middle-income countries. In terms of geographical spread, almost two-thirds of studies (n=57) covered a country in sub-Saharan Africa. This was followed by South Asia (n=26) and only a small number of studies covered the Latin America and the Caribbean region (n=5), East Asia and Pacific (n=2) and Europe and Central Asia (n=1).
Supplemental material
Most education and training interventions made use of smartphones (n=65). This was followed by basic/feature phones (n=48) and tablet devices (n=11). Cases where there was overlapping use of smartphones and basic phones can be explained by use of applications on both types of phones. Interventions were often implemented by a mix of organisations (n=34) but also researchers and NGOs led intervention implementation too (n=25/n=19). Four programmes were implemented by private sector organisations, and only two were implemented exclusively by government agencies.
The interventions assessed in the included studies focused on a diverse range of CHWs. Applying inductive coding, we identified a total of 28 different terms to refer to CHWs in the included studies (online supplementary appendix 5). ‘Community health worker’, unsurprisingly, topped this list (n=46) followed by ‘accredited social health activist’ (n=14). ‘Frontline health worker’ and ‘health surveillance assistant’ were used in three studies each.
Supplemental material
In terms of study design, a total of 38 evaluations were identified. These comprised of Randomised Control Trials (RCTs) (n=7), quasi-experimental designs (n=22) and process evaluations (n=9). A second large group of 27 studies applied more descriptive study designs such as descriptive surveys, case studies and assessments. The third large group of studies referred to qualitative study designs (n=16). This group included designs such as qualitative case studies (n=8), ethnographies (n=3) and action research (n=3). Only four studies made use of explicit mixed-methods designs.
Mapping the evidence
The heatmap in figure 2 provides a visual overview of the patterns in the evidence-base. On the heatmap, the green colour spectrum indicates a larger size of evidence, with the red colour spectrum indicating a smaller size of the evidence. The heatmap thereby indicates the size of the evidence and its gaps regarding different configurations of training approaches and health conditions. For example, the heatmap shows a strong cluster of evidence on the use of mobiles to support ‘work-force management and planning’ (n=29) but there is no evidence at all on ‘supporting supervision’ of CHWs in the area of mental health (n=0). We discuss the results and interpretation of this evidence map more in Discussion section.
We also visualised the mapped evidence-base on an interactive evidence mapping tool: https://africacentreforevidence.org/project-outputs-6/. The screenshots below illustrate how this tool can be used by researchers, policymakers and practitioners. The interactive evidence map applies the same mapping framework as shown in the heatmap. The size of the bubbles indicates the size of the available evidence-base (figure 3); the larger the bubble, the larger the evidence-base. Users can access the evidence contained in each bubble directly by clicking on it. This then displays a list of the relevant primary studies (figure 4). Where full texts of studies are available, users can access these directly as part of the meta-information displayed for each study (figure 5). Furthermore, users can also apply a range of filters (ie, region, type of CHWs, type of implementation, type of technology and type of study design) to tailor-make and customise the evidence map (figure 6). This allows users, for example, to only explore the evidence-base for their region or to only see certain types of evidence.
Overview of the evidence-base through an interactive evidence map. CHWs, community health workers; LMICs, low- and middle-income countries.
Accessing primary studies throughout the interactive evidence map.
Meta-information recorded about each study and link to full texts.
Using filters to customise the evidence-base.
Discussion
The heatmap (shown in figure 2) shows clear differences in the use of mobile technologies in training across the evidence-base. The number of occurrences (ie, the row and column sums) indicates the total evidence-base for the type of training (column total) and health condition targeted (row total). This number of occurrences is a more comprehensive indicator of the coverage than the total number of studies. This is because most studies addressed programmes that combined a range of training approaches (eg, an application that supports work-force management and serves as a job-aid tool at the same time) and, likewise, addressed multiple health conditions. The total number of studies per health condition or per training type does not capture this correlation. In all, 53 studies covered more than one health condition (eg, seven studies covered HIV and infectious disease). In total, 36 studies covered more than one training approach (eg, 11 studies covered workplace management and planning and supervision). Figures 7 and 8 further contextualise the evidence-base by providing details of the total number of studies for each variable. For example, in figures 7, 48 studies focused on maternal health alone but when the total evidence-base is considered, we find 70 occurrences.
Types of health conditions targeted in the evidence-base.
Types of training targeted in the evidence-base. CHW, community health worker.
Overall, we can say that there is an insufficient evidence-base to inform policy or practice as 37 out of the 70 areas (or 53%) covered by the evidence map only have a single (18%) or no study at all (35%). In particular, many areas with a high global burden of disease19 show little or no coverage. Nutrition, disability and trauma are severely neglected, with five, three and one occurrence, respectively. Non-communicable diseases (12 occurrences), infectious disease (11 occurrences) and mental health (10 occurrences) show only low-to-moderate coverage. HIV/AIDS has only moderate coverage (20 occurrences). What is clear is that the evidence based is skewed towards a cluster of evidence on maternal health and child health (71oc and 52oc currencies, respectively)(in 30 studies, the intervention targeted both maternal and child health), both existing priority areas in global health. While we can’t say definitively why this might be the case but the clustered nature of the map indicates that funding for training interventions follows existing priorities.
With respect to the training methods used, there is a very clear divide between those that are workplace-based and those that used more traditional forms of pedagogy (figure 8). Workplace management and planning (76 occurrences) and on-the-job aids (69 occurrences) are used in all nine areas of global health making them by far the most common forms of training (in 20 studies, the applied intervention combined both these approaches). This could be a reflection of the importance placed on training CHWs to follow protocols on their phones (eg, Integrated Community Case Management (ICCM) apps). It may also be because this type of intervention has measurable outcomes when project managers gain increased oversight of relatively large numbers of (often highly dispersed) CHWs. However, this approach to accountability may happen at the expense of increased CHW autonomy, leading to a situation where CHWs do not fully reflect on their own practices.6
There is then a steep decrease down to approaches that draw on more traditional forms of training: supervision (19 occurrences), information dissemination (12 occurrences), formal training (11 occurrences) and peer learning (seven occurrences). However, although small, the evidence map shows the growing importance of social forms of learning for CHWs (eg, supervision and peer learning). From an educational perspective, these have strong potential as they can be designed to complement workplace-based learning. While this is a welcome development, we must remain cautious as the evidence map says nothing about the quality and nature of social learning practices. Indeed, evidence for supportive supervision remains weak.8 9 In a previous scoping review,21 we discussed the importance of supportive supervision as part of the diverse range of ongoing training options and recommended that training needed to be designed from a health systems strengthening perspective, in line with our previous work.22
While interactive messaging and voice systems still prevail in provider-to-provider communication (15 occurrences), an interesting finding revealed by the evidence map is the beginnings of a shift away from information dissemination as the dominant training modality. This could indicate that the availability of smartphones is overcoming the long-held and leading role played by basic/feature phones in CHW training. Smartphones are now the most popular technology used in training programmes (n=65), which is reflective of their downward trajectory in cost over the last few years in LMICs. While this seems to have opened up opportunities to design programmes that engage in forms of social learning, there remains much to do: For example, 14/19 occurrences of supervision were found in maternal health and child health, and peer learning remains the least used form of training.
More worryingly, there seems to be little appetite for developing training in areas of neglect. There are no studies that use information dissemination, peer learning or supervision in the areas of nutrition, trauma, disability or mental health. We believe that developing training opportunities in these areas is a great opportunity to address the health needs of the marginalised poor.
Conclusion
While research has investigated the evidence-base for using mobile technologies in global health, evidence for their use in training of CHWs has remained comparatively overlooked. The evidence map presented in this paper has produced a mixed picture. On the positive side, social learning is no longer the preserve of training programmes aimed at doctors and nurses but is now beginning to be actively designed and implemented for CHWs. On the other hand, programmes are focusing on relatively well-resourced areas of global health, potentially reinforcing existing vertical implementation strategies that have resulted in siloed practice. However, the evidence map shows that this is partially compensated for by the implementation of training programmes that addressed more than one area of global health.
Overall, the evidence map has shown that very clear gaps remain regarding the choice of health condition targeted for training and in the type of training used. Effectively tackling this issue will require an interdisciplinary approach to research and practice with a deeper concern for evidencing the work of CHWs who care for the most marginalised members in their communities. Significant work will be needed to improve the breadth and depth of the evidence-base: first, foundational research on assessing the quality of mobile-based training programmes in terms of their role in improving the capabilities of CHWs is urgently required. Second, programme design and implementation strategies must result in more robust evidence on how mobile technology can support social learning. This will enable the development of new insights in CHW practice, for example, using technology as part of a mixed-methods approach to gather data on learning practices in support of assessment and evaluation.
Acknowledgments
We thank the reviewers for their insightful comments on the paper.
Footnotes
NW and LL are joint first authors.
Handling editor Kerry Scott
Contributors Background framing and establishment of the research question: NW and LL. Mapping framework: All authors. Development of search strategy: LL, CP, AG, SN and NW. Database search: LL, PN and JR. Record screening: PN, JR and JOD; Coding and data extraction: PN, JR, LL and JOD. Evidence Mapping Visualisation: LL. Discussion: All authors. Conclusion: NW and LL. First draft: NW and LL. All authors provided comments on drafts of the article and approved the final version for publication.
Funding This research was funded by the University of Oxford under the HEFCE-GCRF support fund, project number: 0005267. The funder had no role in the study design, data collection, data analysis or writing of the manuscript. JO is a DPhil candidate at The University of Oxford and is supported by a grant from the Economic and Social Research Council (ES/P000649/1). SN is funded through a Medical Research Council Clinical Training Fellowship.
Competing interests None declared.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement All data relevant to the study are included in the article or uploaded as supplementary information.