• health policy
• health systems
• health systems evaluation
• indices of health and disease and standardisation of rates
• public health

The Global Health Security Index (GHSI) claims to be ‘the first comprehensive assessment and benchmarking of health security and related capabilities across the 195 countries that make up the States Parties to the International Health Regulations (IHR)’.1 It is funded by the Open Philanthropy Project, the Bill and Melinda Gates Foundation and the Robertson Foundation. The work itself was conducted by the Nuclear Threat Initiative and Johns Hopkins School of Public Health in conjunction with the Economist Intelligence Unit, hereafter referred to as ‘the GHSI team’. Amidst the several other existing global health security assessment tools available, we critically review whether the GHSI adds value to the existing suite of tools for improved global health security.

Evaluation of compliance with the legal requirements of the IHR2 was initially based exclusively on country self-reporting. Assessments were undertaken using the IHR Self-Assessment Annual Reporting (SPAR) tool.3 Following the West African Ebola epidemic in 2014–2016, recommendations were made by the IHR review committee to strengthen assurance of IHR compliance.4 In 2015, the 68th World Health Assembly recommended5 expanding the approach to assessing IHR compliance and developed the Monitoring and Evaluation Framework.6 The Joint External Evaluation (JEE) tool was developed to provide a more transparent, independent and objective assessment of a country’s ability to deal with health security concerns. The current JEE tool (second edition)7 is a set of 49 indicators under 19 technical areas in order to establish a baseline assessment, with assessments occurring every 4–5 years. This is now included in a suite of preparedness assessment tools that include the SPAR, simulation exercises and After Action Reviews. In 2019, the GHSI emerged as an additional assessment of a country’s global health security. This comprised a spreadsheet-based tool that captures all the results, an accompanying summary report and articles highlighting various findings, all of which are available on the GHSI website.1 According to the accompanying report,1 ‘the GHSI adds health system resilience, compliance with international norms, and risk environments to the JEE’s foundational assessments of prevention, detection, and response’.

The GHSI assessment is based on open-source information for each of the 195 IHR signatory countries on how they meet 85 subindicators across six categories. To conduct the assessment, the GHSI team searched for documents related to these indicators, with each country being given the opportunity to comment via their US Embassy or United Nations Mission, although at the time of this writing only 16 countries have responded. The GHSI is comprehensive, encompassing a broad range of factors relating to global health security, from a number of healthcare professionals to urban development. There are however some examples of what could be perceived as a skew of priorities towards elements of global health security deemed important in high-income settings within the GHSI. The varying context and needs of individual health systems are treated with its one-size-fits-all approach. For instance, the GHSI team specifically mention the JEE’s lack of Global Catastrophic Biological Risks as an issue despite the existence of biosafety and biosecurity indicators in the JEE (JEE indicators P.6.1 and P.6.24). The emphasis placed on biosafety, while important, is not where an under-resourced healthcare system dealing with a large burden of endemic infectious diseases can devote a lot of resource. A low-income and middle-income country (LMIC) would have a lower score as a result, but may actually be devoting its resources appropriately to its own priorities.

Each country’s compliance with the indicators can be explored individually, or compared with specific countries, regions, income groups, population sizes and whether they have completed a JEE or not, within the GHSI tool that captures all the results. This use of open-source data is to encourage transparency and accountability—characteristics that a health system should aspire to. However, while transparency is important, one criticism of the GHSI approach is that it neglects the realities of some LMICs as key documents are not necessarily accessible or available publicly. As a result, countries that have better capabilities could be scored lower if the relevant documents were not made available, potentially biasing the results.

The scoring system allows for granularity since scores are allocated to each subindicator and subsequently weighted to generate a quantitative value for each indicator and subindicator. The weighting assigned to each subindicator and indicator to generate the overall score is adjustable—which is a welcome layer of flexibility. However, the scoring system across indicators is not consistent and questionable; some indicators can be assessed as either 0 or 100, whereas others can use the entire spectrum of numbers in between 0 and 100. For instance, indicator 3.2 (exercising response plans) can either be scored as 0, 50 or 100, whereas indicator 3.6 (access to communications infrastructure) appears to have dozens of possible scores between 0 and 100. Likewise, assigning any form of arbitrary weighting could be viewed as problematic, as global health security priorities across the world are not uniform. For example, zoonotic disease (indicator 1.2) is assigned 1.7% more weight than antimicrobial resistance (indicator 1.1) using the expert weighting. The comparative value of these two issues is context-specific and subjective. Caution must be taken when interpreting and comparing the scoring of indicators and scoring across countries. There is a risk of reductionist use of comparative simple percentages by policymakers and healthcare professionals alike.

Some indicators may be questioned regarding their validity. For instance, urbanisation, while certainly posing public health issues, is a subindicator (indicator 6.4.1a). Therefore, simply by having a more urbanised population alone, a lower score would be given. This does not consider how the country is dealing with the health issues arising from urbanisation; instead this approach assigns a score to an issue which cannot directly be influenced by the country’s health policymakers. Finally of particular concern is the tendency towards a simplistic ‘league table’ approach to comparing countries that the visual approach to presenting the GHSI enables. High-level summaries, maps and plots can be generated to compare selected countries or regions, giving GHSI outputs a visual appeal. We have observed a pattern of countries ranking themselves by GHSI score, without consideration of the variability and nuance within the indicator calculations as described above.

The authors question whether the GHSI tool provides any new value to the existing global health security assessment tools, given the aforementioned ways in which scoring and comparisons could be misleading and the level of resource required in producing the GHSI. The GHSI paints a broad picture of where global health security stands, where improvements could be made, and the differences between countries with varying income levels, population sizes and regions. The data collection for each country is impressive and could be used for country-level gap analysis, identifying specific gaps where scores are low in a particular domain. However, ranking countries based on weighted scores across indicators that are scored variably and are not directly comparable with one another is problematic. Change in score over time for each individual country may be a more useful way of using this information, to track progress and impact of national public health institutes, projects and partnerships. We recommend avoiding using the scoring to determine priorities and compare countries with one another. We look forward to further refinement of the process and suggest closer integration with the JEE process and IHR.