Unanswered questions and remaining challenges
Sheikhtaheri et al
31 describe many challenges with designing and implementing expert systems for supporting clinical decision making. In general, they conclude that the successful implementation of any expert system requires a clear definition of the clinical problem to be addressed. Building and updating the knowledge base is challenging in the best of circumstances and would be compounded in resource-poor settings. Many expert systems also lack an accuracy tracking mechanism, which could undermine the trust of clinicians and patients. Sheikhtaheri et al
31 also raise interesting questions of whether all disease domains require expert systems and how various AI systems should be integrated. They suggest that the lack of answers for these questions has prevented expert systems from advancing from research to implementation.
Supervised machine learning applications require high-quality datasets that can be used to train machine learning algorithms to identify risk factors or make disease diagnoses.31 For many diseases and conditions relevant to resource-poor settings, such datasets can be difficult and time-consuming to collect.31 In addition, better diagnosis does not equate to access to appropriate or quality treatment options. While remote diagnostics and machine learning applications might help to identify diseases, there is no guarantee that the condition identified can be treated in any given setting. As such, the principle of ‘do no harm’ and the ethics around providing treatment following testing for and confirmation of disease (‘test and treat’) when deploying AI applications are relevant as much as in any other context.
Carrel et al
32 describe some of the challenges associated with adapting clinical NLP systems to diverse healthcare settings. Using colonoscopy screening as an example, they highlight the substantial resources necessary to compile the natural language corpora, address different record structures and deal with idiosyncratic linguistic content. These challenges would likely be multiplied when considering healthcare settings in low-income countries. Countries in low-resource settings sometimes maintain hand-written health records in local languages. Therefore, building the natural language corpora could require substantial effort. The WHO has advocated for the adoption of standardised medical terminologies or the development of local data dictionaries to address some of these challenges.33
Substantial data are necessary to build and implement automated planning and scheduling applications.29 Compiling such data in resource-poor settings is often difficult and time-consuming. Brunskill and Lesh describe the extensive data collection involved in laying the groundwork for the development of an improved CHW schedule in sub-Saharan Africa.29 Furthermore, the effectiveness of automated planning and scheduling applications will depend largely on the quality of data used to develop the application. High-quality health systems data are currently difficult to collect in many resource-poor settings. Such challenges could slow the development of automated planning and scheduling in the settings where they would be most needed.
While internet connectivity is improving throughout the world, some resource-poor settings remain without access to the substantial bandwidth necessary to upload very large signal datasets to the cloud. Some applications, including mHealth tools used by CHW, have the ability to work offline and sync with remote databases when the bandwidth is sufficient. Storing such data locally could also require substantial investment in IT infrastructure. While device prices are currently also a barrier, these are predicted to come down over time as companies take advantage of economies of scale.
There are also environmental challenges that need to be considered. The use of AI in resource-poor settings require a strong understanding of local social contexts, infrastructure requirements and additional related infrastructure needs, including IT, communications networks and platforms for delivering primary health services. Many AI applications depend on the availability of strong electronic health record systems, which require substantial investment to put into place. In addition, AI applications will have limited impact if they do not effectively integrate languages and scripts used in the electronic health records of many developing countries.
In high-income countries, discussions around the ethics of electronic health records and AI have focused largely on privacy, confidentiality, data security, informed consent and data ownership. Most of these same considerations apply to resource-poor settings. However, the relevance of these issues varies depending on differences in culture, literacy, patient–provider relationships, available IT infrastructure and regulatory issues in LMICs. One proposed approach for maintaining secure and transparent health records is the use of ‘blockchain’, the distributed ledger system known primarily for its use by crypto-currencies.
In addition, some experts have raised concerns that some AI applications can potentially exacerbate inequities, including those related to ethnic, socioeconomic and gender. They note that cultural prejudices can be reflected in data, algorithms and other aspects of AI design.34 One recent report found that an application that uses arrest records, postal codes and socioeconomic data to assess the risk of recidivism in US courts was biased against black citizens.35 These challenges are compounded by the fact that many AI algorithms are a ‘black box’ and are therefore less likely to be assessed for bias. However, some researchers are working to assess biases by testing how well they predict by randomly changing key variables for individuals for whom AI applications are attempting to make a prediction.36 For AI to benefit all, including those in resource-poor settings, these biases need to be considered in the design of such applications. In health, ensuring that more women and those resource-poor communities are involved in the development of AI applications will also help to reduce such biases.
The generation of large amounts of data naturally raises questions around the ownership of the data and who can access which specific data for research or commercial purposes. While there have been some initial efforts to address this, including the Data Sharing Principles in Developing Countries, which was put forward in Nairobi in 2014, there has yet to be widespread adoption.37 The underlying concept that initiated the development of these principles is that data generated with public funds should be viewed as a public good. Establishing a repository for large amounts of data for global health would help in ensuring that such data are made available as a global public good. Many LMICs have signed agreements that data generated with use of public funds should be freely available.37 However, both from patient and developer perspectives, privacy laws and data access and ownership agreements are perceived to be potential threats to successful AI applications and they should therefore be monitored closely by groups working to develop such applications for particular contexts. Applications using new technologies like blockchain, may also help resolve some of the current concerns.
