Article Text
PDF
Abstract
Introduction Information systems for community health have become increasingly sophisticated and evidence-based in the last decade and they are now the most widely used health information systems in many low-income and middle-income countries. This study aimed to establish consensus regarding key features and interoperability priorities for community health information systems (CHISs).
Methods A Delphi study was conducted among a systematically selected panel of CHIS experts. This impressive pool of experts represented a range of leading global health institutions, with gender and regional balance as well as diversity in their areas of expertise. Through five rounds of iterative surveys and follow-up interviews, the experts established a high degree of consensus. We supplemented the Delphi study findings with a series of focus group discussions with 10 community health worker (CHW) leaders.
Results CHISs today are expected to adapt to a wide range of local contextual requirements and to support and improve care delivery. While once associated with a single role type (CHWs), these systems are now expected to engage other end users, including patients, supervisors, clinicians and data managers. Of 30 WHO-classified digital health interventions for care providers, experts identified 23 (77%) as being important for CHISs. Case management and care coordination features accounted for more than one-third (14 of 37, 38%) of the core features expected of CHISs today, a higher proportion than any other category. The highest priority use cases for interoperability include CHIS to health management information system monthly reporting and CHIS to electronic medical record referrals.
Conclusion CHISs today are expected to be feature-rich, to support a range of user roles in community health systems, and to be highly adaptable to local contextual requirements. Future interoperability efforts, such as CHISs in general, are expected not only to move data efficiently but to strengthen community health systems in ways that measurably improve care.
- other study design
- global health
Data availability statement
All data relevant to the study are included in the article or uploaded as online supplemental information.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
WHAT IS ALREADY KNOWN ON THIS TOPIC
Evidence exists that highlights how CHISs are being used more and more by health workers across a range of discreet healthcare interventions, expanding in both scope, expected functionality, and user personas, positioning them as complex interventions with an important role to play in strengthening health systems and improving outcomes such as the coverage, quality, speed and equity of healthcare. However, with no shared guidelines on how these tools should be designed and/or speak to other digital platforms, few of these tools are able to make it past the pilot phase or reach scale.
WHAT THIS STUDY ADDS
This study highlights how CHISs—with a strong focus on case management and care coordination—should complement and strengthen health information and reporting systems, such as electronic medical record and aggregate reporting systems. Results also demonstrate that CHISs are expected to be feature-rich, to support a range of user roles in community health systems, and to be highly adaptable to local contextual requirements.
HOW THIS STUDY MIGHT AFFECT RESEARCH, POLICY OR PRACTICE
Future CHIS interoperability efforts are expected not only to move data efficiently, but to strengthen community health systems in ways that measurably improve care. To do so, CHWs should be included in the decision-making processes central to CHIS design, implementation, and evaluation.
Introduction
Nearly half of the world’s population still lacks affordable access to essential health services.1 Health inequities are particularly acute in communities where socioeconomic conditions, humanitarian crises, geography, race, gender and other social determinants present barriers to care.2 To overcome these obstacles, many health systems around the world recruit community members to serve as community health workers (CHWs). Following short preservice or on-the-job medical training, CHWs provide door-to-door care, link people to facilities and offer wraparound social support in contexts where they are typically trusted neighbours and local experts.3
A sizeable body of evidence indicates that CHWs can strengthen primary care systems and significantly improve outcomes for a range of priority health conditions.4–6 Yet the literature also reveals that some large-scale CHW programmes achieve little or no measurable public health impact.7 To achieve consistently good outcomes, CHWs must be supported as part of a functioning community health system.5 8 Strengthening community health delivery entails systematically addressing such issues as supervision, salaries, training, equipment and supply chains, community engagement, referral pathways with facilities, and data feedback loops. In many settings, digital tools have become daily enablers of this full range of community health activities.
Researchers and implementers have been equipping CHWs in low-income and middle-income settings with mobile phones since at least 2008,9–11 and the field has seen rapid growth in the last decade. While early studies were likely to involve text messaging or personal digital assistants, smartphone and tablet-based apps are now commonplace and larger implementations of these platforms support tens to hundreds of thousands of end users. In many countries, community health information systems (CHISs) have significantly more end users than any other health information system, not least of all because CHWs are typically greater in number than other cadres of health workers.
The digital health literature offers some insight into how the common uses of these tools have evolved from ad hoc communication and reporting use cases to a more recent focus on digital interventions that aim to improve such outcomes as coverage, quality, speed or equity of care. Yet it is widely recognised that too many of the rigorously studied digital health efforts have failed to replicate or scale beyond an initial pilot.12 13 By contrast, CHIS deployments that have scaled up significantly, often through collaboration among Ministries of Health and ecosystems of partners, have seldom been systematically described or evaluated in the peer-reviewed literature. This disconnect between research and practice is exacerbated by the fact that community health policy guidance advanced markedly during the decade that CHIS research and practice were taking shape, such that digital health research from even a few years ago may reflect outdated perspectives on community health delivery and vice versa. The result is that a literature review alone would be unlikely to reveal a comprehensive and up-to-date picture of how researchers and expert practitioners expect CHISs to strengthen community health delivery, or what features CHISs commonly employ to do so.
This study began with an interest in exploring the circumstances under which health systems would benefit from CHISs exchanging data with other health information systems. The study team soon discovered that it would be impractical to explore forward-looking interoperability use cases without first establishing a more systematic consensus regarding the aims and features commonly found in CHIS implementations today. The results, summarised in this paper, offer a more comprehensive and evidence-based picture of CHISs than has been available in the peer-reviewed literature to date.
Methods
Delphi study
The Delphi method is a well-established, robust method for building consensus among a group of experts. It is an empirically grounded, iterative technique that typically involves multiple rounds of surveys and follow-up interviews, the opportunity for experts to explain their decisions and revise their opinions with each round, and a degree of anonymity and structure so as to avoid groupthink and direct confrontation among experts.14–16 The Delphi method has been widely used in information systems and health research, and a number of studies have shown that for questions requiring expert judgement, the average of individual responses is inferior to the averages produced by group decision processes such as the Delphi method.17
Following the example of prior Delphi studies, we opted for a mixed-methods approach involving quantitative and qualitative data. Qualitative methods can be helpful in cases where it is important to elicit and parse out nuanced information not normally gained through quantitative methods,18 particularly when individual opinions and knowledge will be selected, compared and combined in order to address a lack of agreement or an incomplete state of knowledge.19 20 Our surveys included structured data collection (eg, Likert scales and ranking/prioritising) and qualitative, open-ended questions.17 We also conducted semistructured interviews (n=9 out of 24 Delphi study panellists) to contextualise data collected from surveys. Our purposive sample of interview participants included (1) experts who responded in surveys that they would be willing to be contacted for follow-up questions; (2) at least one expert from each participant category and (3) consideration of geography and gender representation. For a flow diagram and further explanation of the Delphi process, see online supplemental appendix A.
Supplemental material
Supplemental material
Defining consensus
The primary outcomes of interest for this study were (1) participant responses to open-ended questions and (2) levels of agreement as measured by a concordance threshold, where ‘consensus’ is considered to have been reached when >0.7 (70% of respondents) select ‘strongly agree’ or ‘agree’ in response to a Likert scale question. Experts were not required to reach a minimal threshold of concordance before moving on to a subsequent topic, as non-concordance on topics also represents relevant findings.
Sample size, sampling approach, participant overview and study team
Prior work on the Delphi method emphasises the importance of selecting experts through a rigorous, systematic process.21 There are no clear guidelines on the total number of experts to include in a Delphi panel,22 23 though some have recommended seven per participant group.24 We began this process by assembling a study team with significant experience using all of the most widely implemented CHISs and other digital health systems, including CommCare, Community Health Toolkit, OpenSRP, DHIS2 Aggregate and Tracker, RapidPro, Open Data Kit, and OpenMRS. The study team’s broad network helped us to recruit a diverse and authoritative panel representing the breadth of CHIS expertise globally, rather than drawing expertise primarily from any one company or open-source community’s network. See online supplemental appendix B for details on recruitment via our open call for participants.
A total of 58 applicants submitted a statement of interest and resume and we initially selected 25 participants. One participant withdrew after the first round, leaving 24 panellists who responded to surveys 2–5. This sample allowed for a highly diverse set of perspectives from relevant participant groups, while allowing us to effectively manage the panel of experts and complete analyses of responses within the study’s time frame.
Selection of expert panel
All participants had expertise in at least one area of specialisation: (1) CHIS design and development; (2) CHIS research and implementation; (3) community health research and implementation; (4) digital health or/and community health policy, governance and regulations and (5) funding of digital health and community health programmes. All nominees that met the inclusion criteria were sorted according to their primary areas of specialisation, and all study team members reviewed each nominee’s expression of interest and resume. The study team independently ranked their top (up to) 10 nominees from each specialisation, and then openly reviewed and discussed the rankings until agreement was reached on a final ranking. The study team discussed and took into consideration a range of issues in ranking and selecting panellists: number of years of experience as a designer, practitioner or researcher; relevant experience with CHISs and digital tools; and potential to add to the range of viewpoints sought in the panel. The team also sought to ensure equal representation from specialisations, and to ensure an approximately even number of women and men for the final expert panel, with due consideration to representation from as many different geographical regions as possible. We sent an invitation to participate to the top five ranked participants from each specialisation category. The panel composition, along with their individual affiliations and collective years of experience with designing and implementing community health programmes and/or CHISs are represented in table 1 and figure 1.
Descriptive characteristics of the Delphi panel
Experts opinions about community health system actor roles. CHWs, community health workers; HMIS, health management information system.
Study duration and participant involvement
Data collection lasted approximately 9 months, from February to October 2022. Participants responded to five rounds of questionnaires, for a total average time of just over 3 hours per participant over the study period (median=40 min, first quartile=22.5 min, third quartile=60 min), excluding follow-up interviews that lasted roughly 30 min per participant.
Data analysis
Questionnaire responses were collected from participants via the Google Forms survey software, and data analysis began immediately. The analysis consisted of ranked, descriptive statistics of survey responses and inductive thematic review of qualitative data from surveys and interviews to highlight exemplary and salient quotes.25 26 We presented open-ended data in full to panellists after each survey and presented summary graphics of structured data so that the panel of experts was able to review them during subsequent questionnaires. Alongside each round of questionnaires, the results of the previous round were shared with participants. In cases where structured data demonstrated less than 70% concordance, responses were revisited in the subsequent round and each participant was given the chance to change their previous response. In cases where at least two participants raised a particular topic or theme via open-ended questions, the topic was revisited in a subsequent survey and participants were given the chance to respond. To set a ‘cut-off’ threshold for determining when to close each survey, we used the following criteria: (1) at least one respondent from each participant group and (2) at least 80% (20/24 participants) overall response rate.
Focus group discussions with CHWs
Following the completion of the Delphi process and analysis of results, we conducted focus group discussions (FGDs) with CHWs in order to understand their views of the expert panel’s consensus. We also sought to identify any areas of oversight that might have been less visible to high-level experts and more visible to CHWs, based on their perspective as primary end-users of CHISs. Details regarding sampling, data collection and data analysis for these FGDs are available in online supplemental appendix C.
Results
The response rates for the five surveys were 100%, 100%, 96%, 88% and 83%, respectively. Full surveys can be found in online supplemental appendix D and expanded survey results can be found in online supplemental appendix E. The semistructured interview guide, along with summary notes from the interviews conducted among panellists, is in online supplemental appendix F.
Survey 1: the human and institutional context of CHISs
Prior to the first survey, the study team conducted a preliminary literature review to establish a concise ‘essential reading list’ (see online supplemental appendix G) covering key community health topics that could inform CHIS design and implementation. We asked experts to agree or disagree on whether these documents were indeed essential, and to add to the list to help build a shared repository of formative materials to help guide CHIS designers and implementers.
We also drew on this literature review to select a list of topics that feature prominently in community health research and policy. For clarity, we included quotes to illustrate how the topic has been discussed in the literature and asked experts to agree or disagree on whether each topic refers to an aspect of community health that matters for designers and implementers of CHIS. Experts expressed strong consensus about (1) community health topics, (2) health systems roles and (3) health service areas of relevance to CHISs.
Several participants commented that local health systems staff (eg, district health management teams) play key roles in community health interventions. ‘Even if something is mandated from a national level’, one expert noted, ‘you need the subnational staff to champion the work as well. If they are not supporters of the intervention, it won’t go anywhere.’ In a similar vein, experts recommended involving community stakeholders such as traditional healers, midwives, chiefs and ‘others across different sectors that may have close contact with a family/household (ie, teachers, religious leaders/mentors)’. CHWs are often good sources of information about these other stakeholders, and they can offer unique insights into the daily practicalities of their own work. For example, in our focus groups, they shared observations about the difficulty of paying for data bundles out-of-pocket and the difficulty of mobile apps for CHWs with poor eyesight. Such practicalities are less visible to other stakeholders and were offered as examples of why it is important for CHWs to remain the central stakeholders in CHIS design and implementation.
Survey 2: digital health interventions for community health
The WHO’s classification of digital health interventions (DHIs) categorises various ways that digital technologies are used to support health systems (see online supplemental appendix H for summary figure). This framework aims to promote an accessible and bridging language for health programme planners to articulate functionalities of digital health implementations.
This classification is anchored on the unit of a ‘DHI’, which represents a discrete functionality of a digital technology to achieve health sector objectives. The classification describes how any particular health system challenge might be addressed through a set of DHIs, and these interventions might be supported by one or more types of information systems, such as logistics management information systems, electronic medical record (EMR) systems and CHISs.
While a helpful starting point, the WHO classification does not describe which DHIs are relevant to community health. With this context in mind, the second survey gathered expert opinion on which DHIs are relevant to community health delivery. Below in figure 2 we highlight findings regarding which DHIs for care providers play an important role in CHISs.
Experts’ opinions about digital health interventions for care providers.
In open-ended questions, several panel members questioned whether CHWs should be considered part of the care providers category in the WHO classification. For example, ‘here it was not clear if healthcare provider included the CHW or if it meant the health facility personnel, for example, clinical officer, nurse, physician. I took it to mean the latter’. The WHO classification of DHIs provides only the following concise definition of healthcare providers: ‘Healthcare providers are members of the health workforce who deliver health services’. That said, the classification offers examples of each intervention, and in the care providers section, several of the examples are drawn from community health programmes (eg, see pp. 12–13 of the WHO Recommendations on Digital Interventions for Health System Strengthening, Appendix H). When this topic was revisited in a subsequent survey, 100% of experts confirmed that CHWs should be considered as part of the care providers category in the WHO Classification, and only one participant responded that they might reconsider their responses to survey #2 based on this new information.
We also asked experts to consider any additional DHIs for care providers not captured in the classification that is relevant to community health delivery. Responses included ‘enabl(ing) digital diagnosis and documentation’, ‘providing feedback to healthcare providers on their performance’ and ‘feeding back aggregated health systems performance data to providers’. Figures summarising findings related to DHIs for patients and health system managers can be found in online supplemental appendix E.
Survey 3: common features of CHISs
Widely used CHIS platforms are often highly configurable, with a relatively smaller number of core features that can be tailored to support a large range of digital interventions and health service areas (eg, child health, maternal health). Reusable features such as task and schedule management, messaging and decision support enable software to be ‘tailored to the needs of the end user and application scenario’.27 A feature in this sense is ‘a unit of functionality of a software system that satisfies a requirement, represents a design decision and provides a potential configuration option’.27
To prepare this survey, we reviewed product information and technical documentation pages for CommCare, Community Health Toolkit, OpenSRP, DHIS2 Aggregate and Tracker, RapidPro, Open Data Kit, and OpenMRS, and compiled a list of the terms used to describe the major features of these systems. We then drew on our own working knowledge of these platforms and the wider digital health field to synthesise terms across platforms, and we reworded them with the aim of making feature descriptions generic, rather than particular to any one software platform, health service area or intervention. Participants were then asked Likert scale questions in which they could agree or disagree with the statement that a given feature is important for CHISs.
The results indicate that panel members consider most of the features the survey covered to be important for CHISs. Of the 40 features for which we sought panel members’ input, there was initial consensus that 37 features, or 93%, are important for CHIS. Figure 3 summarises these findings, broken down by category of features for system administration, data collection, care coordination and data management.
Experts’ opinions about features important for CHISs. CHIS, community health information system.
A number of CHIS features for system administration were considered ‘very important, if not essential’, and ‘critical, especially with different users in the system’. One expert commented that these features are ‘critical (for) the assignment (of) role-based access…to ensure users are only able to carry out tasks related to their roles and responsibilities in the system in the care of their patients’. Other features such as managing metadata were considered ‘a good aspiration’ with ‘lots of potential value, but not as important as many other features’.
In the data collection category, there was a lack of any apparent agreement regarding facial recognition technology. While several panel members commented that facial recognition may be an optional ‘nice to have’ feature, others raised ethical questions or described the feature as unnecessary. By contrast, in the case management and care coordination category two messaging features fell just short of consensus. Experts clearly agreed on their value to community health, yet there was dissensus regarding whether they should be internal or external to CHISs (see figure 4). Qualitative responses clarified that the dissensus had to do with perspectives on the software architectures for these specific features. Some viewed interactive voice recording (IVR) and multistep personalised messaging as important to the internal or core feature set for CHISs, while others viewed these as features better suited to external systems, to be accessed by CHISs via interoperability. Well over >70% of experts agreed that these features are important for CHISs to use by one or either of these architectures.
Experts’ opinions about interoperability use cases for CHISs. CHISs, community health information systems. IVR stands for interactive voice recording.
Surveys 4 and 5: interoperability use cases for CHISs
Interoperability refers to the ability for one digital tool or platform to communicate with, share data across and/or be functionally compatible with another digital tool or platform. Systems that are not designed for interoperability may still be made to exchange data, but typically this takes more time and effort, because custom code is required to connect or integrate them. By contrast, interoperable systems use standards to streamline the work of exchanging data. When two systems support the same interoperability standard, they typically are expected to require little or no custom software development to be able to exchange data.
In theory, CHISs might be expected to exchange data with dozens of other types of health information systems. In practice, a relatively smaller set of interoperability scenarios or use cases have been seen as higher priority and explored more extensively in active implementations. Survey 4 initially covered eleven interoperability use cases that the study team drew from an OpenHIE technical working group focused on interoperability standards for CHISs (This working group was initially formed with a grant from Digital Square and led by Medic with participation from Dimagi, Ona and Accenture. More information is available at https://wiki.ohie.org/display/SUB/Community+Health+Worker+Community+of+Practice).
Four interoperability use cases fell just below or just above the initial consensus threshold. Specifically (1) 68.2% agreed or strongly agreed for ‘anonymous or tracked entity event reports’; (2) 68.2% agreed or strongly agreed for ‘bidirectional duplication of programmatic patient data between CHIS and EMR’; (3) 72.7% agreed or strongly agreed regarding the priority of ‘syncing vital registration data between CHIS and a Master Person Index’ and (4) 77.3% agreed or strongly agreed regarding ‘automate assigning patients to CHW catchments’. All four use cases were revisited in survey 5, with Likert scale questions presented alongside qualitative responses that experts had offered as reasons for agreeing or disagreeing. In survey 5, all four of these use cases fell short of the consensus threshold.
Survey 5 also considered two additional use cases, each of which had been mentioned by at least two experts in qualitative responses. When presented with Likert scale questions, only 35% of respondents agreed or strongly agreed that ‘integration of CHIS with health insurance management systems or alternate financing systems’ is a high priority. By contrast, 75% of respondents agreed or strongly agreed that ‘integration of CHIS with stock management systems’ is a high priority interoperability use case. A CHW in our FGDs offered an example of how such an interoperability workflow could be helpful: ‘it will be a good thing if we share the information about the stocks—the drugs… because I’m the one referring patients. I should refer the patient to the hospital knowing that there are drugs’. Panellists were also asked to rank their highest, second highest and third highest priority use cases for interoperability, and the results are visualised below in figure 5.
Experts’ opinions about CHIS features for case management and care coordination. CHIS, community health information system; EMR, electronic medical record; HMIS, health management information system. MPI stands for master patient index.
Discussion
In low-income and middle-income countries, early pilots of digital tools for CHWs were typically narrow in scope and limited to a small range of features. Many of these pilots supported CHWs exclusively and focused on digitising pre-existing paper-based reporting systems for the sake of data completeness or efficiency. The results of this study demonstrate that expectations have evolved considerably over the last decade and a half. CHISs today are expected to be feature-rich, adaptable to a wide range of local contextual requirements, and able to support and improve care delivery. This evolution has repositioned CHISs as complex interventions with a role to play in strengthening health systems and improving outcomes such as the coverage, quality, speed and equity of healthcare.
The growing range of end users for CHISs is a striking feature of this evolution. CHISs were once closely associated with a single role type: the CHW. Our study shows a new expert consensus has emerged; these systems are now expected to engage other end users who play important roles in strengthening community health delivery, including professional and volunteer CHWs as well as supervisors, nurses, monitoring and evaluation staff, and even patients and household caregivers. It is worth noting that in our FGDs, CHWs shared that they often feel forgotten and overlooked in CHIS efforts. Yet in our Delphi panel findings, CHWs clearly are expected to remain the primary users of these systems. Our interpretation of these findings is that CHISs typically are not expected to shift focus to other users and away from CHWs, so much as they are expected to offer CHWs new ways of connecting with additional stakeholders in community health delivery.
The importance of these new role types was clear when we asked experts to map the needs of community health delivery to the WHO Classification of DHIs. Of the classification’s 16 interventions for clients, our expert panel reached consensus that 9 (56%) are important for CHISs. The same was true for 15 of 27 (56%) interventions for health system managers. While many regard CHISs as relatively simpler than hospital-oriented EMR systems, it is notable that experts identified 23 of 30 (77%) interventions for care providers as being important for CHISs. This comparison with EMRs throws into sharp relief the sophistication with which many CHWs are now using digital technology. However, these results should not be interpreted as an indication that CHISs might replace EMRs or vice versa. The key takeaway here for CHIS designers is, as one panellist explained, ‘They should not compete with one another; they are distinct and should complement each other’.
Our investigation of core features of CHISs contributes to the field in two ways. First, we found that case management and care coordination features accounted for a higher proportion of expected CHIS features than any other category. This key finding is a departure from a time when CHISs were regarded primarily as data collection tools, and it also sheds light on how exactly CHIS are expected to support the concrete operational nature of community health work. Recent research has shown the value of tailoring CHISs to support the operational tasks that are unique to the work of providing care door to door or wherever patients may be found, especially in remote or rural settings.28 CHWs interviewed in our study affirmed this point, citing several technical challenges that CHIS designers might avoid through close consultation with CHWs about the details of their daily activities.
Second, the high degree of consensus we found for a number of CHIS features suggests that these features are common enough across CHIS platforms to merit standards and interoperability efforts regarding the data generated by these particular features. A feature may look or perform somewhat differently from one software system to another. Moreover, some systems have unique features, and product development teams may limit a product’s feature set for a range of compelling user experience and economic reasons (ie, more is not necessarily better). That said, when some features are developed in a relatively common way across platforms, these platforms have an opportunity to align on standardised data structures. Shared data structures can be valuable because they make it much easier for two digital health systems to reliably exchange data, with a range of potential benefits for patients and health systems.
Having established consensus regarding features that are both common and valuable in CHISs, we were able to consider what kinds of data these features are typically generating, and what interoperability use cases may be possible. While our panel of experts reached consensus regarding the potential value of nine interoperability use cases, qualitative responses made it clear that these use cases are not expected to be of equal practical importance in the near future. In surveys and interviews experts shared a range of reasons for prioritising some interoperability use cases over others, related, for example, to technical feasibility or complexity, readiness of external systems to exchange data with a CHIS, potential for clinical or public health impact, potential to improve the care experience for patients and household members, potential to realise efficiencies or improve the health worker experience, and potential to streamline management of health systems.
Funding remained a significant and enduring obstacle, as did the absence of enabling policies. In the words of one panellist, ‘A lot of countries do not have a digital strategy funded, and even if they do, CHISs may or may not be a priority’. Despite these uncertainties, expert responses to our fourth and fifth survey offer a clear sense of priorities and potential next steps for CHIS designers and implementers interested in working on CHIS interoperability projects. Few studies have yet described such interoperability projects in LMICs, let alone rigorously evaluated whether or how such projects might strengthen community health delivery. Given the significant level of expert interest in interoperability, this appears to be an important opportunity for practical demonstrations and further research. Furthermore, involving CHWs who have experience using CHISs may help elucidate potential interoperability challenges, such as the data privacy and sharing issues which were raised in our FGDs.
A key overarching finding of this study, relevant to all stakeholders, is that the inclusion of CHWs in CHIS design and decision-making processes is crucial for the advancement of the field. CHWs can offer detailed insights about their work and communities. They often attend to challenges of significant practical importance, such as poor eyesight or burdensome out-of-pocket expenses (see online supplemental appendix D), which may not occupy the foremost attention of international scientists, engineers and policy experts. We should note that while our FGDs aimed to encompass a range of geographies, our sample of CHWs was limited to those working in eastern and southern sub-Saharan Africa. Future work engaging CHWs and other local experts to assess how this study’s results relate to the particular aspirations and constraints of diverse contexts seems highly warranted. Indeed, 100% of our Delphi panel agreed or strongly agreed that ‘local context and human-centred design’ is a community health topic that matters for designers and implementers of CHISs. As one expert frankly noted in response to a question about CHIS implementation challenges and possibilities, ‘context is everything’. It is the hope of the authors that this study’s findings will be interpreted in this light.
Limitations
There is always a risk of ‘homophily bias’ in studies with methodologies that purposively select participants. This study is no different; though we believe we used a systematic approach to both select a diverse group of participants and to conduct the study. Not including CHWs as part of the study team or having CHWs on the expert panel is another limitation of this study.
Conclusion
The Delphi method’s systematic approach to consensus building and the authoritative calibre of this panel of experts establish these findings as a highly credible basis for collaborative technology development and policy work regarding CHIS in general, and standards and interoperability in particular. An important part of the value of this Delphi study is that it offers a forward-looking view of CHISs, including their core features and priority use cases for interoperability. Using mixed methods, such an analysis of expert consensus offers a helpful complement to literature reviews, which necessarily focus more on prior research than on contemporary practice. We also incorporated the views of CHW leaders, who are often missing from policy-oriented studies. The importance of involving CHWs in future CHIS design and implementation efforts was emphasised by the CHWs we interviewed as well as every single member of the Delphi panel.
This study’s panel of experts clearly expect interoperability to play an important role in CHIS design and implementation in the coming years, yet there remains a degree of scepticism regarding interoperability efforts that for the most part have yet to yield positive results in routine CHIS implementations. This suggests a need for practical demonstration projects, and the diverse range of expert comments regarding such demonstrations is perhaps best summarised by reference to the wider evolution of expectations regarding CHISs in general. The transition to more flexible and feature-rich CHISs comes with an expectation that these systems not only move data efficiently, but that they strengthen community health delivery in ways that measurably improve the coverage, quality, speed or equity of care. This Delphi study provides an evidence-based consensus as a starting point for CHIS designers and implementers as they undertake interoperability projects with these important aspirations in mind.
Data availability statement
All data relevant to the study are included in the article or uploaded as online supplemental information.
Ethics statements
Patient consent for publication
Ethics approval
This study involves human participants, but the study received an IRB exemption from the University of Washington’s Human Subjects Division (ID# 00011919) and was approved by PATH’s Scientific Research Committee (ID# RES-00080). Participants gave informed consent to participate in the study before taking part.
Acknowledgments
We gratefully acknowledge the participants in our expert panel. While several of these experts chose to remain anonymous, the list of experts who chose to share their names and affiliations can be found in online supplemental appendix C. We would also like to thank two anonymous reviewers who provided valuable input to strengthen this paper.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Handling editor Valery Ridde
Contributors IH and MB conceived of the Delphi study. IH and DC designed the initial study protocol and BD, BW, DC, IH, MA, MB and NP contributed to protocol refinement. IH, DC and MA drafted Delphi surveys and BD, BW, DC, IH, MA, MB and NP contributed to refining surveys. MA conducted quantitative analysis and DC conducted qualitative analysis of survey data. MA, MB, NP, BD and DC conducted and analysed semistructured interviews. JO'D designed and conducted the focus group discussion, and JO'D and MB analysed FGD results. IH and DC wrote the manuscript draft. All authors (DC, IH, MA, MB, NP, BD, JO'D and BW) edited and revised the manuscript draft. All authors (DC, IH, MA, MB, NP, BD, JO'D and BW) reviewed and approved the final manuscript draft. IH serves as the guarantor of the manuscript.
Funding This study emerged from a Technical Working Group on standards for Community Health Information Systems, formed as part of the Open Health Information Exchange community with the support of a grant from Digital Square (2134-01705554-SUB) and the Rockefeller Foundation (2019 HTH 022). This working group was led by Medic with participation from Dimagi, Ona, The Community Health Impact Coalition and Accenture, with additional collaborators from The University of Washington joining as coauthors for this study. We are also grateful for the financial support provided by the Patrick J. McGovern Foundation (Grant # N/A) to publish this paper in an open access journal.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.