Article Text

‘Imperfect but useful’: pandemic response in the Global South can benefit from greater use of mathematical modelling
  1. Sandip Mandal1,
  2. Kanchan Parchani2,
  3. Nimalan Arinaminpathy2,
  4. Swarup Sarkar3,
  5. Balram Bhargava4,
  6. Samiran Panda1
  1. 1 Division of Epidemiology and Communicable Diseases, Indian Council of Medical Research, New Delhi, India
  2. 2 MRC Centre for Global Infectious Disease Analysis, and the Abdul Latif Jameel Institute for Disease and Emergency Analytics, Imperial College, London, UK
  3. 3 School of Public Health and Community Medicine, University of Gothenburg, Goteborg, Sweden
  4. 4 Indian Council of Medical Research, New Delhi, India
  1. Correspondence to Dr Samiran Panda; pandasamiran{at}gmail.com

Abstract

Mathematical modelling has been a helpful resource for planning public health responses to COVID-19. However, there is a need to improve the accessibility of models built within country contexts in the Global South. Immediately following the overwhelming ‘second wave’ of COVID-19 in India, we developed a user-friendly, web-based modelling simulator in partnership with the public health experts and health administrators for subnational planning. The purpose was to help policy-makers and programme officials at the state and district levels, to construct model-based scenarios for a possible third wave. Here, we describe our experiences of developing and deploying the simulator and propose the following recommendations for future such initiatives: early preparation will be the key for pandemic management planning, including establishment of networks with potential simulator users. Ideally, this preparedness should be conducted during ‘peace time’, and coordinated by agencies such as WHO. Second, flexible modelling frameworks will be needed, to respond rapidly to future emergencies as the precise nature of any pandemic is impossible to predict. Modelling resources will, therefore, need to be rapidly adaptable to respond as soon as a novel pathogen emerges. Third, limitations of modelling must be communicated clearly and consistently to end users. Finally, systematic mechanisms are required for monitoring the use of models in decision making, which will help in providing modelling support to those local authorities who may benefit most from it. Overall, these lessons from India can be relevant for other countries in the South-Asian-Region, to incorporate modelling resources into their pandemic preparedness planning.

  • COVID-19
  • mathematical modelling
  • control strategies
  • health policy
  • health services research

Data availability statement

Data sharing not applicable as no datasets generated and/or analysed for this study.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

SUMMARY BOX

  • Mathematical modelling has played an important role in the global response to COVID-19, particularly in making important projections pertaining to potential care needs. Close integration of modelling into decision making has so far occurred principally in countries in the Global North, such as the USA and the UK. Some examples of modelling from the Global South include the South African COVID-19 modelling consortium, and the COVID-19 International Modelling Consortium. A particular value will be approached where model development is led by in-country experts, ensuring that the model remains locally relevant.

  • We describe a recent experience from India, of developing and deploying a user-friendly modelling ’simulator’, led by scientists, public health experts and health administrators including Union and State Health Secretaries and National Health Mission Directors in India. Following the overwhelming ‘second wave’ of COVID-19 in India, the purpose of the simulator was to allow health officials and policy-makers in various states in India to model plausible scenarios for a third wave.

  • If modelling resources are to be useful for decision making, it is critical to build them with local data and with country ownership. We make several recommendations for maximising the utility of modelling in pandemic preparedness in the Global South. For example, our experiences highlight the need to establish modelling resources and collaborative networks during ‘peace time’, ready to be deployed at short notice during any future public health emergency.

Introduction

Mathematical modelling—for all its limitations—can serve as a useful tool for planning public health responses to COVID-19.1 The World Health Organization (WHO) has documented numerous examples of modelling being used to support decision-making during the pandemic.2 Nonetheless, it is noticeable that the close integration of modelling into decision-making has so far occurred principally in countries in the Global North, such as the USA and the UK. If the need to protect scarce healthcare resources is one of the critical objectives of modelling, this need is felt even more urgently in the Global South than elsewhere. Some examples of modelling from the Global South include: the South African COVID-19 modelling consortium, which reports directly to public health authorities in the country3; and the COVID-19 International Modelling Consortium, which at the time of writing has been made available to 40 countries.4 Here, we present a case study from India where similar modelling resources were recently built, for states and districts across the country. The model is named ‘CHROMIC’ (for Collaborative Health Research On Modelling-Indian Council of Medical Research (ICMR) & Imperial College). Like other model applications, this model included a user-friendly interface, for policy-makers to interact directly with the model. However, a unique aspect of this modelling tool is that it was developed and delivered by modellers within India, working with collaborators from abroad. The model thus benefited from locally relevant data, as well as a close understanding of the needs of the healthcare system. In what follows, we describe our experiences in the development and deployment of this modelling resource in India. Although we outline some technical details of the model, our focus in this narrative is on how future such initiatives can be made most useful for decision-makers. Based on our experience, we make recommendations for the use of such initiatives in pandemic preparedness in the Global South.

Impetus for the modelling resource

Following a relatively mild ‘first wave’ of COVID-19, which peaked in September, 2020, India witnessed an overwhelming onslaught of the Delta variant of the virus. This second wave, which reached its greatest height in early May 2021, had four times the peak reported cases as in the first wave.5 In a country as large and diverse as India, these events highlighted the need for strategic planning regarding resource allocation and infrastructure strengthening. As part of such preparedness, ICMR, the apex institute for biomedical investigation (research), led a collaboration with Imperial College London to develop a model of SARS-CoV-2 transmission in the country6–10 (see online supplemental figure S1, supporting information for an outline of the model). In June 2021, immediately following the decline of the second wave, the CHROMIC model was used to examine whether a third wave in India could be as severe as the second wave. In brief, results highlighted that such an outcome would occur if: (1) a new variant emerges that shows full immune escape from previously circulating variants (ie, against severe outcomes as well as infection), along with substantially increased transmissibility or (2) lockdowns in specific local areas showing high levels of transmission were suddenly relaxed. The subsequent emergence of the omicron wave validated the findings of this study: at its peak in January 2022, reported cases were two-thirds that of the peak during the second wave. While Omicron showed substantial immune escape allowing widespread reinfection, emerging evidence from other settings showed that prior immunity remained effective against severe disease, hospitalisation and death.11–13 Thus, the omicron wave was far milder than the preceding delta wave, in terms of demand for hospital-based care and mortality.

Supplemental material

However, at the time of publication in June 2021 of the original model (which itself was developed in February 2021), none of this was apparent. The possibility of a severe third wave remained. While the published study focused on the national level, it was recognised that states and districts would benefit from the use of this model for their own planning, using locally relevant data. In particular, we aimed to develop a user-friendly, web-based ‘simulator’ through which state-level planners could interact with the model, with minimal training in modelling techniques (see figure 1 for an example screen from the simulator) and handful of input parameters.

Figure 1

Illustrative screenshot of the simulator. The left-hand panel includes all user-specified scenario specifications. The central panel shows model projections, including the first two waves, while the right-hand panel shows model projections for required hospital capacity in the third wave.

The aim of the simulator—developed through consultation with stakeholders, as described below—was to make projections for the hospital capacity that would be required in the event of a third wave. Given the uncertainty at the time on how a third wave might emerge, the simulator allowed users to specify different scenarios for mechanisms, including (1) the waning of immunity to previously circulating strains, (2) the transmissibility of any future variant, (3) the degree of immune escape of any such variant and (4) the release of local lockdowns and other restrictions in spite of emerging infections. The simulator also allowed users to specify scenarios for ramping up vaccination coverage, to simulate ways of mitigating the impact of a potential future wave. Importantly, the simulator-generated outputs for resource requirements drew heavily from the countrywide COVID-19 Registry14 and were therefore grounded in country-context and reality. The model underlying the simulator was written in Python, an open-source programming language. The simulator was developed through collaboration with Dure technologies, a locally present (India-based) consultancy firm (available here, https://covidwaveapp.duredemos.com). Overall, such three-way collaboration was unique in several respects; first, it involved close partnership between Government, academia and the private sector (Department of Health Research—ICMR, Imperial College and Dure technologies, respectively). Second, the model underlying the simulator was led by the scientists at ICMR, with support from Imperial College, ensuring that the ownership of the model and validation remains in the country in which it was to be deployed.

Participatory development of the simulator

Preparatory discussions: It was important to engage with relevant stakeholders from the beginning of development of the simulator, to ensure that they would have maximal input into its design and functionality. Early discussions involved senior policy-makers, health administrators and programme planners: we presented results of the published modelling work, and explained the aim of developing a user-friendly simulator based on this modelling (details in Supporting information), in order to help prepare for a potential third wave. We discussed the priority use cases of such a simulator; for example, whether it should focus on projections for symptomatic incidence, requirements for hospital capacity or the potential impact of increased vaccination, in mitigating health burden. Feedback highlighted that all three would be important, but that the highest priority should be placed on requirements for hospital capacity to mitigate impact while continuing aggressive vaccination. We also received requests for additional scenarios to be incorporated. For example, data from India suggested that not all of those admitted to hospital actually required oxygen support, suggesting that healthcare demand could possibly be better managed by encouraging patients to follow simple measures at community level before seeking hospital admission. We were requested to incorporate the potential impact of such measures, on demand for hospital-based care. Overall, we were advised to prioritise simplicity in the interface, by having a ‘basic and advanced’ input compartment as well as to minimise scientific jargon where possible.

Refinement of the simulator: Once an initial, functioning version of the interface was available, we re-engaged with stakeholders for feedback. Further, we piloted the simulator among a few state health officials. Box 1 shows some example feedback, illustrating the range of topics discussed. We were also advised to make the simulator more accessible for users who were not familiar with model parameters; principally, by providing guidance in pop-up ‘information bubbles’ on what would be considered ‘high’ or ‘low’ values for each of these parameters. One example is the value of Embedded Image for a third wave, reflecting the potential inherent transmissibility of a novel variant. We added help text to explain that ‘If the third wave arises due to a new variant, please enter the transmissibility of such variant as a percentage increase relative to Delta. A value of 0 (zero) means that the new variant is equally infectious to the Delta variant’. Similar, user-friendly guidance was provided for other third-wave parameters in the user’s control, including the rate at which immunity wanes, and the pace of ramping up vaccination coverage.

Box 1

Feedback quotes from potential users and senior administrative officials in India, on the design and functionality of the Collaborative Health Research On Modelling-ICMR & Imperial College (CHROMIC)-simulator

  • ‘Could we have a ‘basic’ section for inputs for projection and another one—maybe advanced inputs related to new variants?’

  • ‘How should we estimate what R0 is in any state or district?’

  • ‘How has the effect of population density been considered in developing this model?’

  • ‘If we do not have SARS-CoV-2 seroprevalence data for a district, what can we use as inputs?’

  • ‘If there are two surges observed during the Delta wave, which value should we use as inputs for the state in the simulator?’

  • ‘If there was no clear-cut lock down, and instead things were done in graded and phased manner, how should we model this using the simulator?’

Another key topic covered in discussions was on the inherent risks of providing a modelling tool as a ‘black box’, in particular the potential for non-expert users to misinterpret model results. One important example was the role of uncertainty; the published version of the model incorporated uncertainty in model parameters, using Markov-Chain Monte-Carlo methods to propagate uncertainty from these inputs into uncertainty in model projections. However, it is important to note that this approach only partially addresses uncertainty. It does not address ‘structural’ uncertainties such as in the natural history of the infection, which would necessitate alternative model structures. Displaying parametric uncertainty in a user-friendly tool would therefore risk giving a false sense of confidence that the underlying model accounted for all sources of uncertainty. Instead, and as a result of these discussions, we adopted a simpler approach to uncertainty, encouraging users to use the simulator to explore different parameters, and to gain an understanding of which parameters matter most for third wave projections. Meanwhile, we would avoid showing parametric uncertainty in model projections, to avoid misinterpretation of these results.

Deployment and training: Once the simulator was completed, all state health authorities in India were encouraged by the union and state health secretaries through an official communication from the Ministry of Health and Family Welfare, Government of India, to use it to generate state-specific projections.15 Interested states were requested to contact ICMR for initial orientation-training on how to use and interpret the model. Participants for this training included potential users from the respective state and district health authorities. The participants mostly tended to be data scientists and epidemiologists, but also included senior public health officials who were interested in gaining familiarity with the simulator’s capabilities. Training, conducted online, began with a brief introduction to what mathematical modelling can and cannot do. It was emphasised that no model is fully predictive, and that models merely make projections on the basis of the best available data. Participants were then shown the simulator, and received a demonstration of how different third wave scenarios could be modelled. They were encouraged to try running the simulator on their own computers, in order to gain hands-on experience.

Uptake of the simulator: Overall, training was attended by over 700 officials from various districts in Indian states. Figure 2 shows outputs from website tracking, produced by Dure technologies, on the number of users visiting the simulator over time; there were 1612 visits to the website, with 1268 unique users as of 24 March 2022. At the central level, projections derived from the simulator helped to formulate travel advisories for known tourist destinations in India. These advisories were developed through a process of engagement of health mission officials of the relevant states.9 However, we caution that—owing to the rapid timeline over which the simulator was developed—it was not possible to instate formal mechanisms for monitoring how states used the simulator, in their decision making. We, therefore, do not have systematic data for the degree to which it influenced state planning, Moreover, and as noted above, the omicron wave in India was far milder, in terms of hospitalisations and deaths, than the delta wave. A more severe third wave would undoubtedly have seen even greater usage of the simulator, for health planning.

Figure 2

Timeline of the number of users of the CHROMIC Simulator for COVID-19.CHROMIC, Collaborative Health Research On Modelling-ICMR & Imperial College.

Recommendations

In the longer term, it is hoped that lessons learnt from this initiative could help to bolster pandemic preparedness in other countries in South and South-East Asia. By design, such preparedness would require enhancement of local capacity in development and use of modelling; critically, this would be focused not only on executing already-available models, but also on developing—and at the very least being able to critique—those models. The need for local district level data cannot be overemphasised as the onset of any wave, spread of infection and availability of health services can widely vary in different parts of a country as large and diverse as India. Thus, it is important for local evidence to be used to help locally relevant decision making. To this end, we offer the following, specific reflections and recommendations, arising from our experience during the COVID-19 pandemic.

Early preparation will be key in future preparedness, including collaborative network. The simulator was developed in a fast-changing pandemic situation, necessitating some compromises in its development. Given more time, we would have preferred to engage with a wider variety of possible users, including not only senior health officials, but also the state-level epidemiologists, data scientists and analysts who would be involved in usage of the tool. Even the simplest models can be complex to interpret correctly. Rather than providing the simulator for unsupported use en masse, in ideal circumstances we would have been able to establish direct rapport with users in each state, and provided follow-up training and guidance where needed. It is not feasible to establish these collaborative networks during emergency response to a pandemic. Instead, it will be important to incorporate modelling support as part of broader pandemic preparedness planning, establishing the necessary capacity and networks in advance of any future pandemic. Organisations such as the WHO, at country, regional and central levels, could play a key role in facilitating the establishment of these networks during ‘peace time’.

Design flexible modelling frameworks to respond rapidly to future emergencies: At a technical level, there are inherent challenges in developing valid models in advance of future pandemics. For example, modelling of COVID-19 was only possible once basic facts about its natural history were known.16–19 One possible approach to address this challenge is to pursue a modular approach, with modelling (‘back-end’) and simulator (‘front-end’) components readily being able to be updated independently. Under such a modular approach, a provisional ‘placeholder’ model—for example, a simple susceptible-exposed-infected-recovered framework—could be developed for the purpose of integration with the simulator. Countrywide engagement would then focus on the required outputs for the simulator (eg, required hospital capacity), as well as developing the network of users described above. Once a future pandemic emerges, how quickly the simulator can be deployed will depend only on how rapidly the model can be updated. Moreover, this model can be further refined as more data comes to light.

Communicating limitations of modelling: While user-friendly tools aim to lower barriers for participating in modelling, it is nonetheless critical for users to understand the limitations of modelling. We found that orientation-training was key for communicating important messages on what modelling can and cannot do, namely, users should be encouraged to actively enquire about the reasons that a given analysis yields the results that it does. Doing so will help immensely in understanding relationships between model assumptions and data on the one hand, and model outputs on the other. As noted above, model projections should never be construed as predictions, but rather as plausible projections based on available data. Furthermore, for any user-friendly modelling tool, there is a trade-off between model complexity and ease-of use. For the simulator, the underlying model is sufficiently straightforward for users to understand scientifically, as well as being computationally faster with the main purpose of planning and preparedness of the health services. Understanding these compromises were important for model transparency among users.

Establish mechanisms for monitoring policy uptake: Not all states would benefit from modelling, depending on their quality of data, existing capacity for implementing user-friendly tools, etc. Even states that would benefit from modelling may need support to maximise its use. To help identify such states, it is important to establish mechanisms for monitoring the use of any simulator in a future pandemic. Here, we have presented website monitoring statistics which, while helpful in informing usage, do not provide information on how the simulator was employed to support planning. In future, addressing this need will be an additional benefit of establishing the collaborative networks described above. As part of this network, users will be encouraged to report on how (if at all) the simulator has been used to inform decision making and, importantly, whether any additional functionality would enhance its value for such purpose.

Conclusions

Overall, user-friendly models can offer helpful tools for local-level planning. Our experience highlights the importance of such modelling approaches having local ownership and relevance, as well as the critical need for users to be equally aware of a model’s capabilities, and its limitations. While no model is perfect, their use can be extremely helpful for national and local authorities in anticipating how best to protect lives and livelihoods from COVID-19.

Data availability statement

Data sharing not applicable as no datasets generated and/or analysed for this study.

Ethics statements

Patient consent for publication

Acknowledgments

We thank Garima Yadav (Data scientist), Roni Thomas (User Interface Design developer) and Nandkishor Bobade (Data Architect) of Dure Technologies Pvt Ltd for developing the web-tool (CHROMIC Simulator) by implementing the model.

References

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Footnotes

  • Handling editor Seye Abimbola

  • Twitter @sandipmathmodel

  • Contributors SP, SS and BB conceptualised the need for a simulator; SM and NA developed the modelling approach and SM performed the modelling, and worked with Dure technologies to develop the simulator. KP helped in data analysis. SP and BB facilitated all engagements with stakeholders to develop the simulator. All authors (analysed) and interpreted the results; NA and SP wrote a first draft of the manuscript, and all authors contributed to the final draft and approved the version for submission to the journal.

  • Funding Authors (SM, BB and SP) acknowledge funding from the Indian Council of Medical Research, and NA acknowledges funding from the UK Medical Research council.

  • Competing interests None declared.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.