Development of the Systems Thinking for Health Actions framework: a literature review and a case study
======================================================================================================

* Jenna Thelen
* Carmen Sant Fruchtman
* Muhammad Bilal
* Kebabonye Gabaake
* Shahid Iqbal
* Tshiamo Keakabetse
* Aku Kwamie
* Ellen Mokalake
* Lucia Mungapeyi Mupara
* Onalenna Seitio-Kgokgwe
* Shamsa Zafar
* Daniel Cobos Muñoz

## Abstract

**Background** Systems thinking is an approach that views systems with a holistic lens, focusing on how components of systems are interconnected. Specifically, the application of systems thinking has proven to be beneficial when applied to health systems. Although there is plenty of theory surrounding systems thinking, there is a gap between the theoretical use of systems thinking and its actual application to tackle health challenges. This study aimed to create a framework to expose systems thinking characteristics in the design and implementation of actions to improve health.

**Methods** A systematised literature review was conducted and a Taxonomy of Systems Thinking Objectives was adapted to develop the new ‘Systems Thinking for Health Actions’ (STHA) framework. The applicability of the framework was tested using the COVID-19 response in Pakistan as a case study.

**Results** The framework identifies six key characteristics of systems thinking: (1) recognising and understanding interconnections and system structure, (2) identifying and understanding feedback, (3) identifying leverage points, (4) understanding dynamic behaviour, (5) using mental models to suggest possible solutions to a problem and (6) creating simulation models to test policies. The STHA framework proved beneficial in identifying systems thinking characteristics in the COVID-19 national health response in Pakistan.

**Conclusion** The proposed framework can provide support for those aiming to applying systems thinking while developing and implementing health actions. We also envision this framework as a retrospective tool that can help assess if systems thinking was applied in health actions.

*   COVID-19
*   Health systems
*   Qualitative study

#### WHAT IS ALREADY KNOWN ON THIS TOPIC

*   Systems thinking is a discipline that has proven useful in addressing problems in health systems strengthening.

#### WHAT THIS STUDY ADDS

*   The Systems Thinking for Health Actions framework simplifies the application of systems thinking in health actions by providing an operational guide.

#### HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

*   This study intends to increase the use of systems thinking implemented in health actions and uncover systems thinking methods that are already being applied in health actions.

## Introduction

Complex adaptive systems (CAS) are systems that contain a myriad of intricately interconnected components. They are dynamic, open systems that change and evolve due to multiple interactions within and across the system, including positive and negative feedback, time delays and tipping points. CAS are self-organising and holistic.1 Health systems can be identified as CAS as they have many interconnected components (ie, agents, such as providers, patients, community, policy makers, and insurance agencies, and structures such as policies, norms, values, histories and capacities) that are constantly changing and adapting to changes.2

In the past 15 years, there have been increasing recommendations to use systems thinking (ST) in health systems because of their complex nature.3 4 ST is a discipline that can support us in making sense of CAS, it focuses on how components of a system are interconnected and how the system behaves.5 6

ST comprised theories, methods and tools that assist with addressing complex problems. It began in the 20th century and has been applied in countless disciplines, including biology, psychology, computer science and anthropology.6 It first emerged as a method for scientific investigation, but in the 1940s, it gained traction as a way to solve real-world problems related to World War II.7 Despite its long history, there is still no single agreed-upon definition of what ST is.8 Forrester and Richmond were among the first to define ST7. In Richmond’s article, we find the first complete definition of ST as ‘the art and science of making reliable inferences about behavior by developing an increasingly deep understanding of underlying structure’.9

Many definitions have followed and they all contain two common attributes: seeing the system holistically beyond just its components, and seeing the components in the context of the whole system.7 In other words, ST focuses on the holistic perspective of a system, and the observed behaviours that emerge from the interactions between the parts of the system.7

Using an ST approach has claimed to be beneficial for understanding and intervening in a health system.3 Trochim *et al*10 have suggested that ST can be used in health systems to create a more holistic view of financing, broaden non-traditional collaborations among disciplines, address the impact of social and political factors and identify barriers to implementing systems approaches.10 ST enables a change in mindset which allows individuals to solve complex problems through a holistic lens.11

Although there is a wealth of theoretical applications of ST in health systems, there is a gap between the conceptual use of ST and its actual application in the real world.12–14 Kwamie *et al*15 suggest that the application of ST needs to be documented better to build a stronger evidence base.15

One practical application of ST in health has been the Systems Thinking for District Health Systems (ST-DHS) initiative, which supported countries and health districts, to apply ST tools and practices to understand and intervene in their local health systems.16 As part of this initiative and to address the gaps in the application of ST in health systems, we have developed the ‘Systems Thinking for Health Actions (STHA)’ framework. This framework provides a structured approach to assessing the extent of application of ST in health actions, where ST terminology may not have been explicitly used. The framework aims to explore the application of ST principles and attributes in the formulation and evaluation of health actions. Furthermore, this new framework intends to be used in health actions as an operational checklist, with ST tools and methods that can be directly applied to the actions.

## Methods

The STHA framework was developed using a combination of a systematised literature review and expert inputs. The developed framework was then applied to a case study within the ST-DHS initiative to explore the ST characteristics in the COVID-19 response in Pakistan.

### Study setting

This research was conducted as part of a bigger project, the ST-DHS initiative, which was implemented in three countries: Botswana, Pakistan and Timor-Leste. The initiative provided two districts per country with ST tools and methods, aiming to improve local health systems with the new knowledge on how to apply ST.16

### Systematised literature review

A systematised literature review was conducted to create the STHA framework. This type of review contains elements of a systematic review but is missing elements that a systematic review would have, such as having two reviewers and registration of the review.17 A systematised review was chosen as time and resources were limited.

We developed a search strategy that included keywords to identify practical and theoretical uses of ST, as seen in online supplemental appendix 1. Using PubMed and Google Scholar, the first 50 results in each search term, based on the best match filter, were assessed for inclusion in the systematised review. The titles and abstracts were independently screened for relevance. If the abstract was pertinent, the full text was read and determined if it was to be included based on the content. Data were extracted and the ST tools and methods used in the manuscripts were compiled into a table (online supplemental appendix 2). They were then categorised into an Xmind map based on their intended uses as stated in the manuscripts.

### Supplementary data

[[bmjgh-2022-010191supp001.pdf]](pending:yes)

### Supplementary data

[[bmjgh-2022-010191supp002.pdf]](pending:yes)

The inclusion criteria were studies that explicitly mentioned and described ST concepts, tools or methods in theory or practice, were available in English and were accessible through the University of Basel. In addition, papers were included if they were published between 1 January 2009 and 31 December 2021. The year 2009 was chosen, as that is the year that ST in health systems gained traction with the publication of *Systems Thinking for Health Systems Strengthening*.3

### Framework development

The aim of the framework was to close the gap between ST theory and application.12–14 Stave and Hopper’s Taxonomy of Systems Thinking Objectives was used as the starting point of the framework. JT, DCM and CSF adapted the taxonomy based on the results of the systematised literature review and developed the first draft of the STHA framework.8 Once the ST characteristics were developed, we included definitions for each of them, as well as categorised the mapped ST tools by characteristic.

The first draft of the STHA framework was presented and discussed at two different virtual participatory workshops consisting of 14 health system researchers with extensive ST experience from Botswana, Pakistan, Switzerland and Timor-Leste. The first workshop consisted of individuals who had participated in the ST-DHS initiative, including the funder. The framework was presented and feedback collected regarding the categories, definitions and categorisation of the tools. The framework was then adapted and a second draft was presented in a workshop with health system researchers based at Swiss Tropical and Public Health Institute (Swiss TPH). The same approach was taken and the feedback was incorporated into a third draft of the framework.

JT, DCM and CSF made the final decision of what to include in the framework. The final draft was shared via email with all participants and those with interest were invited to participate in the write-up of this manuscript (see list of coauthors).

### Pakistan COVID-19 case study

Once we had the final STHA framework, we conducted a case study in Pakistan to validate and test if the framework adequately identified ST in Pakistan’s COVID-19 response. The case study was conducted with key informant interviews and a document review of the National Action Plan for Coronavirus Disease (COVID-19) Pakistan (National Action Plan). The key informant interviews were part of the rapid realist evaluation that was conducted for the ST-DHS initiative evaluation.18 19 The National Action Plan is a document that was developed by the Ministry of National Health Services, Regulation, and Coordination and Government of Pakistan to guarantee that the COVID-19 procedures for outbreak preparedness, containment and mitigation were followed.20 The Pakistan case study was chosen based on convenience as we were able to use the same data that were collected for the ST-DHS initiative.

#### Key informant interviews

The interviewees were two male and one female district health managers from the Islamabad district in Pakistan, representing 38% of total district health managers from the district.

The participants were purposively selected from the ST-DHS initiative and all participants provided consent to the interview. Saturation was discussed and due to the nature of the research, it was determined that three interviews were adequate to test the STHA framework.21 JT interviewed the health managers with whom she had no prior relationship. In addition to JT and the interviewee, a male researcher; MB, from Child Advocacy International, the local research partner, who had been involved in the implementation of the ST-DHS initiative, participated in all three interviews. The interviews lasted between 30 and 45 min and were conducted in English. The interviews were completed over Zoom, recorded with live transcription and stored on the Swiss TPH drive. The interviews were conducted using a semistructured interview guide that was created to evaluate the ST-DHS initiative. The guide was tested before implementation by conducting two practice interviews. Field notes were taken during and after the interviews. The participants did not review the interview quotation table before submitting the manuscript. No interviews were repeated.

### Patient and public involvement

This research did not contain any patient or public involvement.

## Results

### Identification of studies

The initial search of the literature yielded 454 articles, which included four articles from expert input. After removing duplicates and screening the abstracts, 71 articles remained. Following a full-text review, 38 articles met the inclusion criteria and were included in developing the framework (figure 1). The 38 articles comprised systematic reviews (n=2), literature reviews (n=3), qualitative and/or quantitative studies (n=31) and commentaries (n=2). The research was conducted in a multitude of countries, including the USA (n=7), Australia (n=7), Ghana (n=1), Uganda (n=2), Zambia (n=1), Canada (n=1), India (n=1), Pakistan (n=1), Singapore (n=1) and Thailand (n=1). Additionally, there were nine studies with multiple countries and six did not have a specific country of research. The disciplines included in the review were health (n=36), food production (n=1), producing research (n=2), policymaking (n=1) and road traffic safety (n=1). The most commonly used ST tools were causal loop diagrams (n=27), systems dynamics modelling (n=12), agent-based modelling (n=8) and concept mapping (n=6) (table 1).

![Figure 1](http://gh.bmj.com/https://gh.bmj.com/content/bmjgh/8/3/e010191/F1.medium.gif)

[Figure 1](http://gh.bmj.com/content/8/3/e010191/F1)

Figure 1 
Systematised literature review results.

View this table:
[Table 1](http://gh.bmj.com/content/8/3/e010191/T1)

Table 1 
Systems thinking tools

### STHA framework

An iceberg model, shown in figure 2, was chosen to represent this framework, as it allows a perspective shift from the visible health system performance and actions to how the application of ST can unveil underlying structures, patterns and behaviours of the system.22 The ST characteristics are not intended to be considered in order, rather the hierarchy of them only represents the varying levels of technical complexity required to apply them.22 The framework proposes a number of considerations that policy makers, managers, researchers or health practitioners can use to apply ST principles to the design, implementation and evaluation of health actions.

![Figure 2](http://gh.bmj.com/https://gh.bmj.com/content/bmjgh/8/3/e010191/F2.medium.gif)

[Figure 2](http://gh.bmj.com/content/8/3/e010191/F2)

Figure 2 
An iceberg model representing the varying complexity of the six systems thinking characteristics of the STHA framework.

One of the first steps in moving from traditional linear thinking to ST is *recognising and understanding interconnections and system structure*. This characteristic comprised the recognising and understanding that health systems are composed of different interconnected parts. By recognising and understanding interconnections, we are able to gain insight of the key actors within a health system (including the agency of self within the system), as well as the overall system structure. It enables the stakeholders within the system to create shared goals and acknowledge how the health system can work together to implement successful health actions.23

The second ST characteristic is *identifying and understanding feedback in the system* since CAS (as health systems) are governed by feedback.3 Feedback is the cause and effect relationships that occur among the different elements in a system.8 It is critical to identify these relationships, both positive and negative, that occur among the parts of the health systems by recognising feedback loops and determining chains of causality within the system.8 This characteristic builds on recognising interconnections, as it recognises the connections and the directionality between them, and how indirectly an intervention can have a balancing effect on the desired outcome. In Uganda, a causal loop diagram demonstrated how feedback from government restrictions and policies influenced how the dual practice policy developed over time.24 Having an adequate feedback system in health actions allows the changing needs in the system to be identified and allows for interaction and seamless communication among all stakeholders, as well as preventing unintended outcomes.24 25

The third characteristic is *identifying leverage points*, which is a vital characteristic of ST, as these are areas where small changes can have a large impact.26 Identifying leverage points can help determine where to allocate scarce resources in most efficient ways or what small changes in health systems will yield substantial improvements in performance.27 In health systems, identifying leverage points systematically illuminates key areas to intervene, allowing for more targeted health actions.28 Glenn *et al*29 used qualitative data to develop a model of the neglected tropical disease system to identify potential leverage points for eliminating neglected tropical diseases.29

Approaching the more technically complex categories of the STHA framework is the characteristic *understanding dynamic behaviour*. As health systems are CAS, they are non-linear and dynamic over time.29 It is imperative to recognise the feedback loops8 and the interactions between the components of the health system that are responsible for generating patterns of behaviour that can change over time.30 Therefore, recognising dynamic behaviour can help determine the effect that behaviours from components of a health system have on the entire system.8

*Using models to suggest possible solutions to a problem* refers to the use of visualisations to display causality, feedback loops and variables to achieve the purpose of a health system. The visualised models do not depict a real-world system, rather how actors view the system. Developing systems models also include multistakeholder dialogue, which involves the process of defining the problem and delineating the boundaries of the system as well. By depicting the system, stakeholders can gain common understanding of the health system or health actions, as well as each other’s mental models, which facilitates finding potential solutions to the problem.29

Finally, *creating simulation models for testing policies* would arguably be the most complex analysis to integrate ST is *creating simulation models for testing policies*. Using ST simulation models helps translate multifaceted scientific findings into easy-to-understand outcomes.31 Simulation models should combine all previous characteristics and use qualitative and quantitative data to create a comprehensive model of the overall system.8 In health systems, simulation models can be used for assessing vulnerability, economic impact, measuring performance, emergency preparedness and how health systems are interdependent on other systems.32 Using simulation models is an important part of ST as it helps predict the impact a change will have and compare possible solutions to a problem.8

To transform the six ST characteristics into an operational framework for use in health actions, a checklist was created to explore the use of each characteristic in the formulation, design and evaluation of health actions.

The checklist, presented in table 2, shows each of the six framework characteristics in separate categories, as well as the corresponding checklist components and ST tools for each characteristic. The checklist was created as a guide to provoke ideas of how to apply an ST approach. The outlined characteristics and items do not have to be completely checked off to adequately incorporate the ST characteristic in the health action. Additionally, the ST characteristics are not intended to be considered in order.

View this table:
[Table 2](http://gh.bmj.com/content/8/3/e010191/T2)

Table 2 
Systems Thinking for Health Actions checklist

### Application of the STHA framework to the COVID-19 response in Pakistan: a case study

#### Data analysis

The National Action Plan and key informant interviews from the COVID-19 response in Pakistan were analysed using thematic analysis with a deductive approach.33 We applied the STHA framework to analyse the data. First, the interview transcripts and National Action Plan were read and a codebook was created using the STHA framework as a guide. The text was highlighted using the corresponding themes from the codebook. With the relevant areas of the transcript selected, the highlighted text was applied to the six attributes of the framework to determine if and where the selected text fit best. Microsoft Excel was used to manage the data extracted from the transcripts and the data were coded by JT.

The Consolidated Criteria for Reporting Qualitative Research checklist was consulted for reporting this qualitative research.34 The participants gave verbal consent to participate in the interviews, which were recorded.

The National Action Plan *identified the system structure* by listing the key stakeholders and sectors (pp 18–20) involved in the COVID-19 response (including those outside the health system) and the actions expected to be taken by each one of them (p 104). Furthermore, the National Action Plan also textually showed the *connections and emergent behaviour between the components of the system* through ‘rapidly establishing and strengthening coordination to deliver strategic, technical and operational support through existing mechanisms and country partnerships’ (p 11) and creating a ‘policy framework for federal, provincial and regional stakeholders for building capacity to prevent, detect and respond to any events due to COVID-2019 or other novel pathogens with pandemic potential in Pakistan’ (p 9). The action plan also recognised the need for stakeholder involvement and included the relevant stakeholders in the COVID-19 surveillance system (p 14) and the development of the Risk Communication and Community Engagement initiative (p 17).

Respondents 2 and 3 identified the use of process mapping in the district, which assisted with identifying the stakeholders involved in the system. Respondent 3 mentioned the value of mapping stakeholder connections and roles to see how they impact each other. Additionally, respondents 2 and 3 also described identifying *emergent behaviour* in the system using reflective practice.

*To identify and understand feedback* the COVID-19 action plan implemented a monitoring and evaluation plan for constant improvement of the COVID-19 response (p 20). This included a parallel evaluation to continually point out areas for improvement. Additionally, the action plan identified the effect that one component of the system (specifically funding) can have on other components by acknowledging the roles that funding has on surveillance structures, data, laboratory diagnostic capacity, case management, stockpiling and logistics, infection prevention and control, burial policy and risk communication (p 17).

Respondent 3 explained how *understanding the connections* between the components of the process map assisted in realising the effect one component of the system has on another: ‘[process mapping] *gave a very clear pattern of how things were how many stakeholders were involved in everything, and the reflective processes, has had a lot as well you know, to be honest.*’

The COVID-19 action plan in Pakistan *identified the gaps and leverage points* in their COVID-19 system response. ‘Assessments of risks and capacities to determine priorities for emergency preparedness’ were conducted (p 12). Among the gaps mentioned in the action plan were the capacities of case management, risk communication and infection prevention and control at health facilities (p 13), and the disease outbreak management system needing to be strengthened (p 14). The action plan also identified 19 key action areas or leverage points where actions could help minimise the spread of COVID-19 (pp 20–31).

The action plan also *addressed the system’s dynamic behaviour* by predicting how preparedness in the initial phase and strict containment in the second phase would determine the impact of the virus (p 8). It also mentioned ‘strengthening and reforms of the organizational, structural and coordination mechanisms to ensure the maximum level of preparedness over time’ (p 10).

In summary, four out of the six characteristics of the STHA framework were identified in Pakistan’s COVID-19 response.

## Discussion

ST has been a commonly used approach in various disciplines to address multifaceted problems in CAS.6 Adopting an ST approach is an attractive method in the field of health systems, but there is still a lack of understanding of the practical uses of ST.35 This framework aimed to bridge the gap between theoretical ST and practical ST.13 33 Six key ST characteristics were identified in the framework: (1) recognising and understanding interconnections and system structure, (2) identifying and understanding feedback, (3) identifying leverage points, (4) understanding dynamic behaviour, (5) using mental models to suggest possible solutions to a problem and (6) creating simulation models to test policies. We identified two potential applications for the framework and checklist: (1) Prospectively, to support in the design or implementation of health actions. Applying the framework prospectively can be done as a guide to translate ST concepts into practical steps that can be integrated in the design or implementation of a health action. (2) Retrospectively, to investigate where ST was applied and where it can be further applied the next time. The case study provided in this article is a retrospective example of applying the STHA framework.

We developed the STHA checklist to act as a guide to explore the application of ST prospectively or retrospectively in health actions. Checklists ease work in demanding or tense situations and have been increasingly used in healthcare.36 Checklists help promote active cooperation and communication among stakeholders.37 Therefore, using our checklist to assist in translating the application of ST concepts into practical steps can be beneficial. The checklist, which should not be taken as a strict and linear document, can help relieve some of the barriers to applying ST, such as many stakeholders understanding that ST requires sophisticated and resource-intensive interventions, as well as the lack of knowledge on how to start using ST.

The application of the STHA framework to the COVID-19 response in Pakistan revealed that the less complex characteristics were applied throughout the response despite the exact ST terminology not necessarily being mentioned. In our interviews, we identified several ST tools being used by district health officials, such as reflective practice or process mapping.38 The district officials and the research team used these tools as part of the ST-DHS initiative. In addition to the use of ST tools, we also identified other ST characteristics in the COVID-19 response (such as the understanding of dynamic behaviour in the National Action Plan) that were not influenced by the ST-DHS initiative. This highlights how ST is often used without being explicitly mentioned. Given the high complexity and changing environment of the pandemic, the health officials had to apply holistic responses.

The COVID-19 pandemic presented a crisis where policy makers had to quickly respond to a threat without knowing the exact extent of it.35 Therefore, the lack of application of the other three categories in the framework could be due to the time-sensitive nature of the pandemic and the lack of time between creating policies and implementing them. In previous literature, several barriers have been described to the application of ST approaches such as assumed costliness, lack of understanding, competing political interests across health and non-health actors, which often lead to prioritisation of vertical programmes, work in silos and difficulty ensuring meaningful multistakeholder involvement.39 Although the framework cannot mend all of these perceived barriers, it can assist in informing health action creators with more knowledge of using ST tools. Additionally, by revealing that ST exists in many health actions, it can show that the use of ST does not have to be costly, as it is already being applied to health actions without further costs.

In the Pakistan case study, we were able to validate the framework and determine that the framework was effective in identifying ST characteristics in the action plan and key informant interviews. However, sometimes it was difficult to determine if a section of text adequately included an ST characteristic. To lessen these uncertainties, further defining the characteristics will ease the use of the checklist in the future. Additionally, the checklist was only applied retrospectively to assure its use in assisting with the application of ST in impending health actions, the framework should also be validated in prospective cases.

To create the framework, the literature review was not limited to ST in health systems, rather other disciplines were also included (ie, road traffic safety, policymaking and food production). The limitations of this method were that not all disciplines were included in the search, rather disciplines that were previously well known for ST and expert inputs. In addition, the search was limited to English language, manuscripts that could be accessed through the University of Basel, manuscripts that were found on PubMed and Google Scholar and there was only one reviewer. Searching only two academic research databases may have limited the number of results retrieved for the literature review, thereby missing manuscripts that should have been included.40 Therefore, when developing the STHA framework further, a more extensive literature review could be conducted, with the number of disciplines and databases expanded. In addition, expert inputs from other systems thinkers would be helpful to expand the checklist developed for each characteristic of ST. The interviews conducted for testing the framework were specific to the ST-DHS initiative, meaning that the respondents already had knowledge of ST, which could have been a bias in their answers. The framework was also applied to only three interviews, with the respondents having similar positions in the district health system, which may have limited the ideas that interviewing other positions may have added. Expanding the application of the STHA framework to more interviews and with a wider range of positions could assist in further developing it. This was a first pilot of the STHA framework. In order for it to be relevant for a wider set of topics, actions and stakeholders, as well as context, further research is needed to identify how to improve it.

## Conclusion

This paper aimed to bridge the gap between theoretical and practical ST in health actions. Stave and Hopper8 created an excellent starting point for bridging this gap by identifying the level of ST in individuals in their Taxonomy of Systems Thinking Objectives. The STHA framework has made additional progress in closing this gap by creating a tangible checklist for designing, implementing and evaluating health actions. The STHA framework is a new, innovative way to apply ST. This framework can be used retrospectively, and it can be a guide in the development of health actions to explore where ST can still be applied. Further research is needed to ensure the STHA framework reaches its full potential.

## Data availability statement

No data are available.

## Ethics statements

### Patient consent for publication

Not applicable.

### Ethics approval

This study involves human participants and was approved by the Swiss Ethics Committee in Basel and the Health Services Academy in Islamabad, Pakistan. Participants gave informed consent to participate in the study before taking part.

## Acknowledgments

We would like to extend our gratitude to all of the district stakeholders that took part in the Systems Thinking for District Health Systems initiative. We would also like to thank the members of the Systems for Health team at the Swiss TPH for their ongoing contribution to the STHA framework, especially Don de Savigny for his expert input on systems thinking.

## Footnotes

*   Handling editor Stephanie M Topp

*   Twitter @santfro, @Daniel\_Cobos\_PH

*   Contributors JT, CSF and DCM conceptualised the research. JT and MB collected the data for the case study. JT conducted the systematised literature review and analysed the systematised literature review data. JT, DCM and CSF adapted the taxonomy based on the results of the systematised literature review and developed the first draft of the STHA framework. JT applied the framework to the case study and wrote the first draft of the manuscript. All authors reviewed and contributed to the final version. DCM is responsible for the overall content.

*   Funding This work received financial support from the Alliance for Health Policy and Systems Research. The Alliance is able to conduct its work thanks to the commitment and support from a variety of funders. These include our long-term core contributors from national governments and international institutions, as well as designated funding for specific projects within our current priorities. For the full list of Alliance donors, please visit: [https://www.who.int/alliance-hpsr/partners/en/](https://www.who.int/alliance-hpsr/partners/en/)

*   Competing interests None declared.

*   Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

*   Provenance and peer review Not commissioned; externally peer reviewed.

*   Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

[https://creativecommons.org/licenses/by/3.0/igo/](https://creativecommons.org/licenses/by/3.0/igo/)
This is an open access article distributed under the terms of the Creative Commons Attribution IGO License ([CC BY 3.0 IGO](https://creativecommons.org/licenses/by/3.0/igo/)), which permits use, distribution,and reproduction in any medium, provided the original work is properly cited. In any reproduction of this article there should not be any suggestion that WHO or this article endorse any specific organization or products. The use of the WHO logo is not permitted. This notice should be preserved along with the article’s original URL.Disclaimer: The author is a staff member of the World Health Organization. The author alone is responsible for the views expressed in this publication and they do not necessarily represent the views, decisions or policies of the World Health Organization.

## References

1.  Pype P, Mertens F, Helewaut F, et al. Healthcare teams as complex adaptive systems: understanding team behaviour through team members’ perception of interpersonal interaction. BMC Health Serv Res 2018;18:570. [doi:10.1186/s12913-018-3392-3](http://dx.doi.org/10.1186/s12913-018-3392-3)
    
    

2.  Swanson RC, Cattaneo A, Bradley E, et al. Rethinking health systems strengthening: key systems thinking tools and strategies for transformational change. Health Policy Plan 2012;27 Suppl 4(Suppl 4):iv54–61. [doi:10.1093/heapol/czs090](http://dx.doi.org/10.1093/heapol/czs090)
    
    [PubMed](http://gh.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjgh%2F8%2F3%2Fe010191.atom) 

3.  Savigny D, Adam T. Systems thinking for health systems strengthening. In: Alliance for Health Policy and Systems Research. WHO, 2009.
    
    

4.  World Bank. Healthy development: the world bank strategy for health, nutrition, and population results [Internet]. Washington, DC: World Bank; 2007. Available: [https://openknowledge.worldbank.org/handle/10986/6843](https://openknowledge.worldbank.org/handle/10986/6843) [Accessed 8 Mar 2022].
    
    

5.  Hassan I, Obaid F, Ahmed R, et al. A systems thinking approach for responding to the COVID-19 pandemic. East Mediterr Health J 2020;26:872–6. [doi:10.26719/emhj.20.090](http://dx.doi.org/10.26719/emhj.20.090)
    
    

6.  Peters DH. The application of systems thinking in health: why use systems thinking? Health Res Policy Syst 2014;12:51. [doi:10.1186/1478-4505-12-51](http://dx.doi.org/10.1186/1478-4505-12-51)
    
    

7.  Shaked H, Schechter C. Definitions and development of systems thinking. In: Systems Thinking for School Leaders. Cham: Springer, 2017. Available: [https://doi.org/10.1007/978-3-319-53571-5\_25](https://doi.org/10.1007/978-3-319-53571-5_25)
    
    

8.  Stave KA, Hopper M. What constitutes systems thinking: A proposed taxonomy. 25th International Conference of the System Dynamics Society System Dynamics Society; 2007
    
    

9.  Richmond B. Systems thinking/system dynamics: let’s just get on with it. Syst Dyn Rev 1994;10:135–57. [doi:10.1002/sdr.4260100204](http://dx.doi.org/10.1002/sdr.4260100204) Available: [http://doi.wiley.com/10.1002/sdr.v10:2/3](http://doi.wiley.com/10.1002/sdr.v10:2/3)
    
    

10. Trochim WM, Cabrera DA, Milstein B, et al. Practical challenges of systems thinking and modeling in public health. Am J Public Health 2006;96:538–46. [doi:10.2105/AJPH.2005.066001](http://dx.doi.org/10.2105/AJPH.2005.066001)
    
    [CrossRef](http://gh.bmj.com/lookup/external-ref?access_num=10.2105/AJPH.2005.066001&link_type=DOI) 
    
    [PubMed](http://gh.bmj.com/lookup/external-ref?access_num=16449581&link_type=MED&atom=%2Fbmjgh%2F8%2F3%2Fe010191.atom) 
    
    [Web of Science](http://gh.bmj.com/lookup/external-ref?access_num=000235691300028&link_type=ISI) 

11. Adam T, de Savigny D. Systems thinking for strengthening health systems in lmics: need for a paradigm shift. Health Policy Plan 2012;27 Suppl 4:iv1–3. [doi:10.1093/heapol/czs084](http://dx.doi.org/10.1093/heapol/czs084)
    
    [PubMed](http://gh.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjgh%2F8%2F3%2Fe010191.atom) 

12. Systems thinking for noncommunicable disease prevention policy: guidance to bring systems approaches into practice. Copenhagen: WHO Regional Office for Europe; 2022, Licence: CC BY-NC-SA 3.0 IGO.
    
    

13. de Savigny D, Blanchet K, Adam T. Applied systems thinking for health systems research: A methodological handbook. 2017.
    
    

14. Rusoja E, Haynie D, Sievers J, et al. Thinking about complexity in health: a systematic review of the key systems thinking and complexity ideas in health. J Eval Clin Pract 2018;24:600–6. [doi:10.1111/jep.12856](http://dx.doi.org/10.1111/jep.12856)
    
    [PubMed](http://gh.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjgh%2F8%2F3%2Fe010191.atom) 

15. Kwamie A, Ha S, Ghaffar A. Applied systems thinking: unlocking theory, evidence and practice for health policy and systems research. Health Policy Plan 2021;36:1715–7. [doi:10.1093/heapol/czab062](http://dx.doi.org/10.1093/heapol/czab062)
    
    

16. Sant Fruchtman C, Bilal Khalid M, Keakabetse T, et al. Digital communities of practice: one step towards decolonising global health partnerships. BMJ Glob Health 2022;7:e008174. [doi:10.1136/bmjgh-2021-008174](http://dx.doi.org/10.1136/bmjgh-2021-008174)
    
    

17. Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J 2009;26:91–108. [doi:10.1111/j.1471-1842.2009.00848.x](http://dx.doi.org/10.1111/j.1471-1842.2009.00848.x)
    
    [CrossRef](http://gh.bmj.com/lookup/external-ref?access_num=10.1111/j.1471-1842.2009.00848.x&link_type=DOI) 
    
    [PubMed](http://gh.bmj.com/lookup/external-ref?access_num=19490148&link_type=MED&atom=%2Fbmjgh%2F8%2F3%2Fe010191.atom) 

18. Westhorp G. Realist impact evaluation: an introduction. Overseas Development Institute September 12, 2014.
    
    

19. Vindrola-Padros C, Johnson GA. Rapid techniques in qualitative research: a critical review of the literature. Qual Health Res 2020;30:1596–604. [doi:10.1177/1049732320921835](http://dx.doi.org/10.1177/1049732320921835)
    
    [PubMed](http://gh.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjgh%2F8%2F3%2Fe010191.atom) 

20. Ministry of National Health Services. National action plan for coronavirus disease (COVID-19) pakistan. 2020: 136.
    
    

21. Braun V, Clarke V. To saturate or not to saturate? Questioning data saturation as a useful concept for thematic analysis and sample-size rationales. Qualitative Research in Sport, Exercise and Health 2021;13:201–16. [doi:10.1080/2159676X.2019.1704846](http://dx.doi.org/10.1080/2159676X.2019.1704846)
    
    

22. Amran A. Iceberg model [Internet]. 2022. Available: [https://untools.co/iceberg-model](https://untools.co/iceberg-model) [Accessed 1 Jun 2022].
    
    

23. Wilkinson J, Goff M, Rusoja E, et al. The application of systems thinking concepts, methods, and tools to global health practices: an analysis of case studies. J Eval Clin Pract 2018;24:607–18. [doi:10.1111/jep.12842](http://dx.doi.org/10.1111/jep.12842)
    
    

24. Paina L, Bennett S, Ssengooba F, et al. Advancing the application of systems thinking in health: exploring dual practice and its management in Kampala, Uganda. Health Res Policy Syst 2014;12:41. [doi:10.1186/1478-4505-12-41](http://dx.doi.org/10.1186/1478-4505-12-41)
    
    [PubMed](http://gh.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjgh%2F8%2F3%2Fe010191.atom) 

25. Gowda NR, Wankar A, Arya SK, et al. Feedback system in healthcare: the why, what and how. IJMS 2020;12:52. [doi:10.5539/ijms.v12n1p52](http://dx.doi.org/10.5539/ijms.v12n1p52)
    
    

26. Rwashana AS, Nakubulwa S, Nakakeeto-Kijjambu M, et al. Advancing the application of systems thinking in health: understanding the dynamics of neonatal mortality in Uganda. Health Res Policy Syst 2014;12:36. [doi:10.1186/1478-4505-12-36](http://dx.doi.org/10.1186/1478-4505-12-36)
    
    

27. Zukowski N, Davidson S, Yates MJ. Systems approaches to population health in Canada: how have they been applied, and what are the insights and future implications for practice? Can J Public Health 2019;110:741–51. [doi:10.17269/s41997-019-00230-3](http://dx.doi.org/10.17269/s41997-019-00230-3)
    
    

28. Hodges S, Ferreira K, Israel N, et al. Strategies for system of care development: making change in complex systems. 2006;15.
    
    

29. Glenn J, Kamara K, Umar ZA, et al. Applied systems thinking: a viable approach to identify leverage points for accelerating progress towards ending neglected tropical diseases. Health Res Policy Syst 2020;18:56. [doi:10.1186/s12961-020-00570-4](http://dx.doi.org/10.1186/s12961-020-00570-4)
    
    

30. Assaraf OB-Z, Orion N. Development of system thinking skills in the context of earth system education. J Res Sci Teach 2005;42:518–60. [doi:10.1002/tea.20061](http://dx.doi.org/10.1002/tea.20061) Available: [http://doi.wiley.com/10.1002/tea.v42:5](http://doi.wiley.com/10.1002/tea.v42:5)
    
    

31. Powell KE, Kibbe DL, Ferencik R, et al. Systems thinking and simulation modeling to inform childhood obesity policy and practice. Public Health Rep 2017;132(2_suppl):33S–38S. [doi:10.1177/0033354917723601](http://dx.doi.org/10.1177/0033354917723601)
    
    

32. McLean C, Lee YT, Jain S, et al. Modeling and simulation of healthcare systems for homeland security applications NIST IR 7784. Gaithersburg, MD: National Institute of Standards and Technology; 2011. Available: [https://nvlpubs.nist.gov/nistpubs/Legacy/IR/nistir7784.pdf](https://nvlpubs.nist.gov/nistpubs/Legacy/IR/nistir7784.pdf) [Accessed 5 Jun 2022].
    
    

33. Burnard P, Gill P, Stewart K, et al. Analysing and presenting qualitative data. Br Dent J 2008;204:429–32. [doi:10.1038/sj.bdj.2008.292](http://dx.doi.org/10.1038/sj.bdj.2008.292)
    
    [CrossRef](http://gh.bmj.com/lookup/external-ref?access_num=10.1038/sj.bdj.2008.292&link_type=DOI) 
    
    [PubMed](http://gh.bmj.com/lookup/external-ref?access_num=18438371&link_type=MED&atom=%2Fbmjgh%2F8%2F3%2Fe010191.atom) 

34. Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care 2007;19:349–57. [doi:10.1093/intqhc/mzm042](http://dx.doi.org/10.1093/intqhc/mzm042)
    
    [CrossRef](http://gh.bmj.com/lookup/external-ref?access_num=10.1093/intqhc/mzm042&link_type=DOI) 
    
    [PubMed](http://gh.bmj.com/lookup/external-ref?access_num=17872937&link_type=MED&atom=%2Fbmjgh%2F8%2F3%2Fe010191.atom) 
    
    [Web of Science](http://gh.bmj.com/lookup/external-ref?access_num=000251963800004&link_type=ISI) 

35. Bradley DT, Mansouri MA, Kee F, et al. A systems approach to preventing and responding to COVID-19. EClinicalMedicine 2020;21:100325. [doi:10.1016/j.eclinm.2020.100325](http://dx.doi.org/10.1016/j.eclinm.2020.100325)
    
    

36. Greig P, Maloney A, Higham H. Emergencies in general practice: could checklists support teams in stressful situations? Br J Gen Pract 2020;70:304–5. [doi:10.3399/bjgp20X709373](http://dx.doi.org/10.3399/bjgp20X709373)
    
    [FREE Full Text](http://gh.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYmpncCI7czo1OiJyZXNpZCI7czoxMDoiNzAvNjk1LzMwNCI7czo0OiJhdG9tIjtzOjIzOiIvYm1qZ2gvOC8zL2UwMTAxOTEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

37. Grzybowski A, Patryn R, Sak J, et al. Surgical safety checklist. procedure for the safety of procedures. Int J Surg 2017;41:174–5. [doi:10.1016/j.ijsu.2017.03.087](http://dx.doi.org/10.1016/j.ijsu.2017.03.087)
    
    

38. Zafar S, Fruchtman CS, Bilal Khalid M, et al. Lessons learnt of the COVID-19 contact tracing strategy in islamabad capital territory, pakistan using systems thinking processes. Front Public Health 2022;10:909931. [doi:10.3389/fpubh.2022.909931](http://dx.doi.org/10.3389/fpubh.2022.909931)
    
    

39. El-Jardali F, Adam T, Ataya N, et al. Constraints to applying systems thinking concepts in health systems: a regional perspective from surveying stakeholders in eastern Mediterranean countries. Int J Health Policy Manag 2014;3:399–407. [doi:10.15171/ijhpm.2014.124](http://dx.doi.org/10.15171/ijhpm.2014.124)
    
    

40. Bramer WM, Rethlefsen ML, Kleijnen J, et al. Optimal database combinations for literature searches in systematic reviews: a prospective exploratory study. Syst Rev 2017;6:245. [doi:10.1186/s13643-017-0644-y](http://dx.doi.org/10.1186/s13643-017-0644-y)
    
    [PubMed](http://gh.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjgh%2F8%2F3%2Fe010191.atom) 

41. McGill E, Er V, Penney T, et al. Evaluation of public health interventions from a complex systems perspective: a research methods review. Soc Sci Med 2021;272:113697. [doi:10.1016/j.socscimed.2021.113697](http://dx.doi.org/10.1016/j.socscimed.2021.113697)
    
    [PubMed](http://gh.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjgh%2F8%2F3%2Fe010191.atom) 

42. Karim A, Cobos Munoz D, Mäusezahl D, et al. A systems approach to assessing complexity in health interventions: an effectiveness decay model for integrated community case management. Glob Health Action 2020;13:1794106. [doi:10.1080/16549716.2020.1794106](http://dx.doi.org/10.1080/16549716.2020.1794106)
    
    

43. Owen B, Brown AD, Kuhlberg J, et al. Understanding a successful obesity prevention initiative in children under 5 from a systems perspective. PLoS ONE 2018;13:e0195141. [doi:10.1371/journal.pone.0195141](http://dx.doi.org/10.1371/journal.pone.0195141)
    
    

44. Kwamie A, van Dijk H, Agyepong IA. Advancing the application of systems thinking in health: realist evaluation of the leadership development programme for district manager decision-making in Ghana. Health Res Policy Syst 2014;12:29. [doi:10.1186/1478-4505-12-29](http://dx.doi.org/10.1186/1478-4505-12-29)
    
    [PubMed](http://gh.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjgh%2F8%2F3%2Fe010191.atom) 

45. Varghese J, Kutty VR, Paina L, et al. Advancing the application of systems thinking in health: understanding the growing complexity governing immunization services in Kerala, India. Health Res Policy Syst 2014;12:47. [doi:10.1186/1478-4505-12-47](http://dx.doi.org/10.1186/1478-4505-12-47)
    
    

46. Bensberg M, Allender S, Sacks G. Building a systems thinking prevention workforce. Health Promot J Austr 2020;31:436–46. [doi:10.1002/hpja.325](http://dx.doi.org/10.1002/hpja.325)
    
    [PubMed](http://gh.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjgh%2F8%2F3%2Fe010191.atom) 

47. Agyepong IA, Aryeetey GC, Nonvignon J, et al. Advancing the application of systems thinking in health: provider payment and service supply behaviour and incentives in the Ghana National health insurance scheme -- a systems approach. Health Res Policy Syst 2014;12:35. [doi:10.1186/1478-4505-12-35](http://dx.doi.org/10.1186/1478-4505-12-35)
    
    [CrossRef](http://gh.bmj.com/lookup/external-ref?access_num=10.1186/1478-4505-12-35&link_type=DOI) 
    
    [PubMed](http://gh.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjgh%2F8%2F3%2Fe010191.atom) 

48. Hamdani Y, Jetha A, Norman C. Systems thinking perspectives applied to healthcare transition for youth with disabilities: a paradigm shift for practice, policy and research. Child Care Health Dev 2011;37:806–14. [doi:10.1111/j.1365-2214.2011.01313.x](http://dx.doi.org/10.1111/j.1365-2214.2011.01313.x)
    
    [CrossRef](http://gh.bmj.com/lookup/external-ref?access_num=10.1111/j.1365-2214.2011.01313.x&link_type=DOI) 
    
    [PubMed](http://gh.bmj.com/lookup/external-ref?access_num=22007980&link_type=MED&atom=%2Fbmjgh%2F8%2F3%2Fe010191.atom) 
    
    [Web of Science](http://gh.bmj.com/lookup/external-ref?access_num=000296911100007&link_type=ISI) 

49. Schoenenberger LK, Bayer S, Ansah JP, et al. Emergency department crowding in singapore: insights from a systems thinking approach. SAGE Open Med 2016;4:2050312116671953. [doi:10.1177/2050312116671953](http://dx.doi.org/10.1177/2050312116671953)
    
    

50. Clarke B, Kwon J, Swinburn B, et al. Understanding the dynamics of obesity prevention policy decision-making using a systems perspective: a case study of healthy together Victoria. PLoS One 2021;16:e0245535. [doi:10.1371/journal.pone.0245535](http://dx.doi.org/10.1371/journal.pone.0245535)
    
    

51. Friel S, Pescud M, Malbon E, et al. Using systems science to understand the determinants of inequities in healthy eating. PLOS ONE 2017;12:e0188872. [doi:10.1371/journal.pone.0188872](http://dx.doi.org/10.1371/journal.pone.0188872)
    
    

52. Knai C, Petticrew M, Mays N, et al. Systems thinking as a framework for analyzing commercial determinants of health. Milbank Q 2018;96:472–98. [doi:10.1111/1468-0009.12339](http://dx.doi.org/10.1111/1468-0009.12339)
    
    [CrossRef](http://gh.bmj.com/lookup/external-ref?access_num=10.1111/1468-0009.12339&link_type=DOI) 
    
    [PubMed](http://gh.bmj.com/lookup/external-ref?access_num=30277610&link_type=MED&atom=%2Fbmjgh%2F8%2F3%2Fe010191.atom) 

53. Mutale W, Ayles H, Bond V, et al. Application of systems thinking: 12-month postintervention evaluation of a complex health system intervention in Zambia: the case of the BHOMA. J Eval Clin Pract 2017;23:439–52. [doi:10.1111/jep.12354](http://dx.doi.org/10.1111/jep.12354)
    
    

54. Kunpeuk W, Teekasap P, Kosiyaporn H, et al. Understanding the problem of access to public health insurance schemes among Cross-border migrants in Thailand through systems thinking. Int J Environ Res Public Health 2020;17:5113. [doi:10.3390/ijerph17145113](http://dx.doi.org/10.3390/ijerph17145113)
    
    

55. Pocock NS, Chan Z, Loganathan T, et al. Moving towards culturally competent health systems for migrants? applying systems thinking in a qualitative study in malaysia and thailand. PLoS ONE 2020;15:e0231154. [doi:10.1371/journal.pone.0231154](http://dx.doi.org/10.1371/journal.pone.0231154)
    
    

56. Brereton CF, Jagals P. Applications of systems science to understand and manage multiple influences within children’s environmental health in least developed countries: a causal loop diagram approach. Int J Environ Res Public Health 2021;18:3010. [doi:10.3390/ijerph18063010](http://dx.doi.org/10.3390/ijerph18063010)
    
    

57. BeLue R, Carmack C, Myers KR, et al. Systems thinking tools as applied to community-based participatory research: a case study. Health Educ Behav 2012;39:745–51. [doi:10.1177/1090198111430708](http://dx.doi.org/10.1177/1090198111430708)
    
    [CrossRef](http://gh.bmj.com/lookup/external-ref?access_num=10.1177/1090198111430708&link_type=DOI) 
    
    [PubMed](http://gh.bmj.com/lookup/external-ref?access_num=22467637&link_type=MED&atom=%2Fbmjgh%2F8%2F3%2Fe010191.atom) 

58. Rosas SR. Systems thinking and complexity: considerations for health promoting schools. Health Promot Int 2017;32:301–11. [doi:10.1093/heapro/dav109](http://dx.doi.org/10.1093/heapro/dav109)
    
    [PubMed](http://gh.bmj.com/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fbmjgh%2F8%2F3%2Fe010191.atom) 

59. Allender S, Brown AD, Bolton KA, et al. Translating systems thinking into practice for community action on childhood obesity. Obes Rev 2019;20 Suppl 2(Suppl 2):179–84. [doi:10.1111/obr.12865](http://dx.doi.org/10.1111/obr.12865)
    
    

60. Riley T, Hopkins L, Gomez M, et al. A systems thinking methodology for studying prevention efforts in communities. Syst Pract Action Res 2021;34:555–73. [doi:10.1007/s11213-020-09544-7](http://dx.doi.org/10.1007/s11213-020-09544-7)
    
    

61. Naumann RB, Sandt L, Kumfer W, et al. Systems thinking in the context of road safety: can systems tools help us realize a true “ safe systems ” approach? Curr Epidemiol Rep 2020;7:343–51. [doi:10.1007/s40471-020-00248-z](http://dx.doi.org/10.1007/s40471-020-00248-z)
    
    

62. Xue H, Slivka L, Igusa T, et al. Applications of systems modelling in obesity research: systems modelling review. Obes Rev 2018;19:1293–308. [doi:10.1111/obr.12695](http://dx.doi.org/10.1111/obr.12695)
    
    

63. Minyard KJ, Ferencik R, Ann Phillips M, et al. Using systems thinking in state health policymaking: an educational initiative. Health Syst (Basingstoke) 2014;3:117–23. [doi:10.1057/hs.2013.17](http://dx.doi.org/10.1057/hs.2013.17)
    
    

64. Jetha A, Pransky G, Hettinger LJ. Capturing complexity in work disability research: application of system dynamics modeling methodology. Disabil Rehabil 2016;38:189–94. [doi:10.3109/09638288.2015.1031291](http://dx.doi.org/10.3109/09638288.2015.1031291)
    
    

65. Diez Roux AV. Complex systems thinking and current impasses in health disparities research. Am J Public Health 2011;101:1627–34. [doi:10.2105/AJPH.2011.300149](http://dx.doi.org/10.2105/AJPH.2011.300149)
    
    [CrossRef](http://gh.bmj.com/lookup/external-ref?access_num=10.2105/AJPH.2011.300149&link_type=DOI) 
    
    [PubMed](http://gh.bmj.com/lookup/external-ref?access_num=21778505&link_type=MED&atom=%2Fbmjgh%2F8%2F3%2Fe010191.atom) 
    
    [Web of Science](http://gh.bmj.com/lookup/external-ref?access_num=000294090500016&link_type=ISI) 

66. Broomhead T. Systems science and oral health: implications for dental public health. Community Dent Health, 2019: 55–62.
    
    

67. Carey G, Malbon E, Carey N, et al. Systems science and systems thinking for public health: a systematic review of the field. BMJ Open 2015;5:e009002. [doi:10.1136/bmjopen-2015-009002](http://dx.doi.org/10.1136/bmjopen-2015-009002)
    
    

68. Tracy M, Cerdá M, Keyes KM. Agent-based modeling in public health: current applications and future directions. Annu Rev Public Health 2018;39:77–94. [doi:10.1146/annurev-publhealth-040617-014317](http://dx.doi.org/10.1146/annurev-publhealth-040617-014317)
    
    [CrossRef](http://gh.bmj.com/lookup/external-ref?access_num=10.1146/annurev-publhealth-040617-014317&link_type=DOI) 
    
    [PubMed](http://gh.bmj.com/lookup/external-ref?access_num=29328870&link_type=MED&atom=%2Fbmjgh%2F8%2F3%2Fe010191.atom) 

69. Malik AU, Willis CD, Hamid S, et al. Advancing the application of systems thinking in health: advice seeking behavior among primary health care physicians in Pakistan. Health Res Policy Syst 2014;12:43. [doi:10.1186/1478-4505-12-43](http://dx.doi.org/10.1186/1478-4505-12-43)
    
    

70. Calancie L, Anderson S, Branscomb J, et al. Using behavior over time graphs to Spur systems thinking among public health practitioners. Prev Chronic Dis 2018;15:E16. [doi:10.5888/pcd15.170254](http://dx.doi.org/10.5888/pcd15.170254)
    
    

71. Durski KN, Naidoo D, Singaravelu S, et al. Systems thinking for health emergencies: use of process mapping during outbreak response. BMJ Glob Health 2020;5:e003901. [doi:10.1136/bmjgh-2020-003901](http://dx.doi.org/10.1136/bmjgh-2020-003901)
    
    

72. Conte KP, Davidson S. Using a'‘ich picture'’to facilitate systems thinking in research coproduction. Health Res Policy Sys 2020;18. [doi:10.1186/s12961-019-0514-2](http://dx.doi.org/10.1186/s12961-019-0514-2)