The practical value of the GHS Index

The GHS Index could serve as a powerful tool for measuring and motivating sustainable financing at national, regional and global levels, and its findings could help catalyse political will to fill gaps in health security capacity. The GHS Index also measures indicators directly related to epidemic and pandemic preparedness alongside indicators of broader strengths needed to reinforce health systems, such as political, security and socioeconomic factors that could shape country risk landscapes and capacities to prevent, detect and rapidly respond to outbreaks. Additionally, it complements ongoing efforts to build accountability for national preparedness, such as the Global Preparedness Monitoring Board, the Global Health Security Agenda (GHSA) and the World Bank Health Emergency Preparedness and Response Multi-Donor Fund.5

The GHS Index also aggregates otherwise scattered qualitative and quantitative data into a consolidated, publicly available format that facilitates comparison and monitoring, and provides extensive documentation of expert-vetted health security capacities and capabilities. GHS Index data are thus a valuable complement to after action reviews, JEEs and workshops to develop National Action Plans for Health Security (NAPHS), as well as for drawing comparisons across geographies at the level of both individual questions and categories. With data collection activities pending for the next iteration of the GHS Index, we hope to eventually be able to monitor national, regional and global trends in health security-strengthening efforts over time, identify when countries deprioritise investments and capacities and include new measures of pandemic preparedness informed by global experiences with COVID-19. Globally and nationally, the GHS Index also identifies existing vulnerabilities in preparedness for biological threats. Systematically cataloguing and monitoring these weaknesses could help inform budget allocations and spending decisions, and aid policymakers in setting priorities for health security-strengthening and health systems-strengthening efforts.

Finally, the GHS Index is valuable in its framing of global health security as a multisectoral social project. Effective outbreak prevention, for example, depends in part on addressing root causes of disease emergence, such as those captured in GHS Index indicators examining land use changes, veterinary workforces and agricultural practices. This multisectoral lens—which incorporates measures of country fragility and other political, economic and environmental vulnerabilities—creates a tangible starting point for One Health coordination platforms or national disaster management committees to work across sectors, engage non-traditional partners and formulate multisectoral solutions to complex health challenges. Though processes like JEEs and NAPHS development are intended to be multisectoral, it is the health sector that most often functions as the entry point for other stakeholders and sectors that may not have explicit mandates to strengthen health security.6 The GHS Index thus recognises—and measures—the benefits of whole-of-society approaches to health security that transcend any one sector.

The value of scoring and ranking

Scoring is an intuitive method of summarising complex health system capacities across diverse contexts, monitoring longitudinal and cross-sectional trends and advocating for needed policy change. Scoring is also used widely across many areas of global health, economic, policy and development research, as well as in established health security evaluation tools like the JEE.7–10

In general, scoring is an effective method for measuring, motivating and monitoring efforts to strengthen infectious disease management capacities across high-income, middle-income and low-income settings alike. The Africa Leaders Malaria Alliance, for example, produces scorecards that track country progress in improving financing, capacity-building and increasing coverage of health services for malaria, neglected tropical diseases and malnutrition.11 And, between 2012 and 2016, publication of scores from the Nuclear Security Index (produced by the Nuclear Threat Initiative and developed with EIU) encouraged 10 countries to remove or dispose of highly enriched uranium within their territories.12 The Human Development Index has similarly emerged as a valuable tool for guiding decision making and monitoring policymaking at both national and subnational levels in many countries.8

In this vein, the GHS Index provides high-level composite scores for each country and offers considerable granularity and transparency in scoring justifications at the indicator and sub-indicator levels, with the aim of helping decision makers pinpoint discrete capacities that require strengthening. Notably, GHS Index scores have been used by some countries to create new metrics of health security. For example, Malaysia’s Ministry of Science, Technology and Innovation—in collaboration with PEMANDU Associates and the Sunway Group—has developed a Global COVID-19 Recovery Index (GCI). GCI Scores are calculated from dynamic estimates of confirmed cases and death rates by country (which receive a weight of 70%), along with static measures of prevention capacities (GHS Index, category 1) and public health vulnerability (GHS Index, Indicator 6.5), which receive a weight of 30%.13

Like scoring, ranking is an effective way to galvanise change in low-performing countries and identify outliers, such as countries that score unexpectedly well despite limited resources, or those that rank poorly despite undertaking capacity-building efforts. Outliers, in turn, should prompt deeper analyses of such findings.14 Morse writes, too, that ‘the ‘league table’ style of presenting rankings is…meant to be picked up by the media, and through them the public, non-government organisations, pressure groups, aid agencies and so on, and ultimately provide pressure on decision makers to bring about positive change.’15 Regardless of their overall score and rank, all countries—including high-performing states—should examine specific GHS Index indicators closely as a means of identifying critical weaknesses to address through capacity-building, funding, legislation or other approaches (see the Considerations for GHS Index users section).

A robust, practitioner-informed methodology

Data for each of the GHS Index’s 34 indicators and 85 sub-indicators were either sourced from existing repositories stewarded by reputable groups like WHO, or qualitatively assessed by EIU researchers. Qualitative data were scored on a binary or categorical scale. To minimise researcher bias and subjectivity in scoring, we employed binary indicators wherever possible. Measurements were normalised on a 0–100 scale to facilitate reproducible cross-country, cross-indicator and cross-category comparison. Detailed justifications, sources and scoring criteria are provided for each indicator in the GHS Index model, report and website to ensure both appropriate contextualisation and reproducibility.

Data review and validation comprised a year-long process involving multiple researchers vetting scores, data and justifications in stages to ensure consistency across countries. The model’s default ‘expert’ weights were derived from extensive deliberation and input from the GHS Index International Panel of Experts, a group of 21 subject matter experts in health security representing organizations in 13 countries.16 However, recognising that no single weighting scheme can reflect every country’s priorities, we also built flexibilities into the GHS Index model: users can customise weights based on context-specific considerations, priorities or other criteria, and country scores and rankings adjust accordingly.

Selecting and harmonising metrics of global health security

The GHS Index conceptualises a health security capacity as a state’s ability or potential to carry out a discrete public health or healthcare function necessary to prevent, detect or respond to acute infectious disease threats, be they naturally occurring, accidental or deliberate. Featured indicators in the GHS Index are intended to aid users in monitoring and measuring the investments and processes that enable states to build, sustain and implement these capacities. The GHS Index does not, however, purport to forecast health outcomes or impacts resulting from country investments in strengthening health security capacities.

Some have asserted that the indicators selected for inclusion in the GHS Index may reflect a systematic bias towards higher-income countries.17 Given that national gross domestic product (GDP) and GDP per capita correlate only weakly with overall GHS Index scores—as evidenced by Pearson’s r values of 0.37 and 0.45, respectively—this seems unlikely. Fidler writes, however, that global health security ‘ultimately depends on the quality of national public health systems.’18 As such, we have taken care to include indicators in the GHS Index that measure material determinants of country-level health security capacities, such as preparedness spending—which do correlate strongly with national income.

However, the GHS Index also incorporates social, political, technical and environmental determinants of health security, which do not necessarily correlate directly with national income level alone. On measures of healthcare access (Indicator 4.3), for example, the first-ranking, second-ranking and fifth-ranking countries were Thailand (upper-middle-income), Georgia (lower-middle-income) and Nigeria (lower-middle-income), respectively. Other factors, such as strength of laboratory systems (Indicator 2.1) and international commitments (Indicator 5.3), also correlate more strongly with overall GHS Index scores (r=0.80 and 0.76, respectively) than with overall GDP (r=0.25 and 0.23, respectively). To further prevent potential confounding by income level, users might consider comparing overall, indicator and sub-indicator-level scores across countries within a given income group and adjusting model weights to align more closely with the specific health priorities of a given country or region.

Moreover, indicators were developed with the aim of ensuring the GHS Index’s integration with existing global health security assessment tools and frameworks such as the GHSA, the JEE and the IHR Monitoring and Evaluation Framework, as well as the NAPHS development process, as Razavi et al recommend. However, other scholars have highlighted a need for global health security metrics that do not simply mirror global patterns of wealth distribution, and that more accurately capture health system functionality and performance rather than capacity alone.19 20 Though the GHS Index does extrapolate beyond the metrics featured in the aforementioned frameworks to encompass novel measures of risk, vulnerability and health system readiness, its primary goals remain (1) supporting and enhancing existing health security-strengthening mechanisms in a comprehensive, accessible format; and (2) motivating decision makers in all countries to make needed investments in epidemic and pandemic preparedness.

Thus, while more meaningful metrics of health security capacity are certainly warranted and merit deeper consideration by the international community, we contend that developing these metrics supersedes the original intention of the GHS Index to leverage existing, publicly available data. However, as we learn from COVID-19, we do plan to revise the GHS Index Framework to include new, more targeted measures of pandemic readiness (see the Next steps section).

Consideration of biosecurity, biosafety, health systems and catastrophic threats

The GHS Index treats biosafety and biosecurity as vital components of global health security. As such, the tool includes numerous indicators measuring country capacities in these areas. Admittedly, health priorities in many low/middle-income countries (LMICs) often differ from those of their wealthier counterparts, and many LMICs may have limited capacities to prioritise biosafety and biosecurity.17 However, the health and security risks associated with dual-use research, accidental releases of pathogens and deliberate misuse of biological materials are not equitably distributed across countries. LMICs are more likely to experience significant morbidity and mortality if these threats manifest, and unless steps are taken to monitor and strengthen their respective biosecurity and biosafety capacities, they will remain disproportionately vulnerable.

The GHS Index’s approach to biosecurity and biosafety aligns with that of the IHR (2005), which require core health system competencies that apply universally to all countries regardless of income level. It is worth noting, too, that while category 1 of the GHS Index includes many biosecurity and biosafety indicators, it is not weighted more heavily than other categories: category 1 scores as a whole account for only 16.3% of a country’s total score, while scores for biosecurity and biosafety indicators each determine approximately 2.6% of a country’s overall score. Including these indicators in the GHS Index also aligns with international health security norms, such as those articulated in the IHR (2005), the JEE, the World Organisation for Animal Health’s Performance of Veterinary Services Evaluations, the GHSA, the Biological Weapons Convention and the Cartagena Protocol on Biosafety. The Africa Centres for Disease Control and Prevention has also prioritised these capacities through its continent-wide Initiative on Strengthening Biosafety and Biosecurity, which aligns with both the JEE and IHR implementation.21

The GHS Index also measures domains not explicitly captured in other health security assessment tools, such as health system readiness and preparedness for global catastrophic biological risks (GCBRs). The Johns Hopkins Center for Health Security defines GCBRs as ‘events in which biological agents—whether naturally emerging or reemerging, deliberately created and released or laboratory engineered and escaped—could lead to sudden, extraordinary, widespread disaster beyond the collective capability of national and international governments and the private sector to control.’22 While the JEE’s biosafety and biosecurity indicators are undoubtedly vital metrics of GCBR preparedness, some health system capacities not sufficiently captured in the tool—such as abilities to test, approve and dispense new medical countermeasures—are also essential to GCBR prevention and mitigation. The GHS Index goes above and beyond the JEE in this regard, featuring a dedicated category focusing exclusively on health system capacities required to mitigate epidemic threats. This was the lowest-scoring category across all countries in the GHS Index: 131 countries ranked in the lowest tier of scores, highlighting critical vulnerabilities in global capacities to care for sick patients and protect health workers during public health emergencies, including GCBRs.1 A comprehensive list of GHS Index indicators measuring GCBR preparedness is provided in the inaugural report.1

Transparency and integration with the JEE

Transparency is a cornerstone of global cooperation around health security capacity-building and emergency response. It enables decision makers to track how global health initiatives are financed, detect and respond quickly to emergent outbreaks, coordinate responses with international partners and ensure accountability in public–private partnerships.23–27 COVID-19 has recently reaffirmed the importance of transparency in case reporting, surveillance and containment, especially as countries take steps to resume routine economic, social and educational activities. Transparency in scientific practice has also proven crucial during COVID-19. For example, open exchanges of clinical data, biological samples, genetic sequence data, modelling parameters and assumptions and epidemiological data support evidence-based policymaking around reopening schools and economies, forecasting demand for healthcare services and equitable allocation of scarce resources.28–31

Given the demonstrated importance of transparency in coordinating effective multilateral responses to pandemic threats, the GHS Index also prioritises publicly available evidence of relevant capacities. Thus, countries without publicly documented evidence of these capacities receive lower scores. The drawback of this approach is that a country possessing a given capacity without sharing corresponding evidence via official channels may receive an artificially low score. To reduce the risk of under-scoring, we invited government officials at 195 embassies and missions to the United Nations to review our data with their respective National IHR Focal Points, course-correct our work and share additional information. However, only 16 countries responded to these requests, for which only minimal changes to scoring were necessary.1

The GHS Index is underpinned by the principle that ‘a health threat anywhere is a health threat everywhere,’ an ethos championed by the architects of the GHSA.32 We believe that data describing national health security capacities should be a public good. In this spirit, we anchored the GHS Index in publicly available data and made the tool itself—and all its data—freely available. To ensure alignment with the JEE and integration of all publicly available information shared by countries, the GHS Index also draws heavily from WHO’s repository of publicly available JEE reports. Moving forward, we strongly encourage all countries to document relevant capacities publicly—both in the interest of global cooperation and transparency, and to improve the fidelity of future monitoring efforts.

Leadership, political and security risks, and trust in government

Some experts have rightly observed an inverse relationship between overall GHS Index scores and suboptimal COVID-19 outcomes in some high-income countries—most notably, the USA, which despite having the highest overall GHS Index score, currently reports the highest number of confirmed COVID-19 cases of any country.3 30 33 34 In fact, JEE Scores for both the USA and the UK share the same inverse relationship with reported COVID-19 outcomes.

There are several possible explanations for these observed disparities. For example, while the USA does maintain strong overall capacities for preventing, detecting and responding to epidemic and pandemic threats, ineffective federal leadership and an inability to deploy available capacities may have precluded a concerted whole-of-government response to the pandemic. Still, despite a high overall score, some of the USA’s category-specific scores—specifically, those in category 4 (health system) and category 6 (risk environment)—do, in fact, reflect these vulnerabilities. The USA ranks 175th out of 195 countries on healthcare access, 75th in the world for number of hospital beds and earned a score of only 60.4/100 for clinic, hospital and community care capacity.35 Similarly, the USA ranks 59th out of 195 countries on measures of socioeconomic and political resilience and received a zero on measures of public confidence in government.35 Thus, even in high-scoring countries, weak governance, poor leadership and inconsistent risk messaging could impede effective implementation and coordination of core health security functions during outbreak response. And, as we explain further in the next section, country scores solely reflect the presence or absence of critical health security capacities. They alone do not—and cannot—provide causal explanations for reported health outcomes.

Considerations for GHS Index users

Some researchers have turned to the GHS Index to elucidate observed between-country disparities in COVID-19-associated morbidity, mortality and case fatality estimates.4 30 33 36 However, drawing simple correlations with countries’ overall scores does not account for the fact that these scores are meant to capture capacities spanning the breadth of the health security life cycle, from outbreak prevention at the source, to early detection, to rapid response. In other words, overall scores summarise country capacities across all categories. However, countries with high overall scores may still have low category-level, indicator-level and sub-indicator-level scores that more strongly influence outbreak-associated outcomes. Therefore, we encourage users wishing to analyse correlations between country scores and health outcomes to examine scores at more granular levels, adjust model weights to reflect country contexts and priorities and consider more nuanced outcomes when analysing countries’ performances during outbreaks. For example, users might examine relationships between category 2 (detection) scores and the number of misidentified or under-reported cases in a given country, rather than examining correlations between countries’ overall scores and absolute case counts.

Furthermore, we stress that implementation of health security capacities is rarely—if ever—a linear process. In other words, the mere presence of a given capacity does not always translate predictably into a desired health outcome or impact. Conceptualising health systems as complex adaptive systems may offer more insight into the varied pathways by which countries achieve public health goals. Rickles et al37 note, for example, that single events can have long-lasting effects on complex systems (‘history matters’), a phenomenon known as path dependence. Paina and Peters further explicate this point, describing how path dependence can complicate health system evolution, particularly in the context of rapidly changing technology and when diverse stakeholders are involved—as is the case in the global health security space.38 They also highlight the futility of transposing health system innovations from countries where they have succeeded into countries with different political processes, or that have not established the institutions or systems required for said innovations to succeed. This might further explain why countries with similar levels of capacity (as indicated by GHS Index scores) might nevertheless report divergent COVID-19 outcomes.

Composite scores and quantitative correlations are thus useful, but alone are insufficient tools for determining causal relationships between health system capacities and observed health outcomes. The GHS Index’s transparent accounting of country capacities across various domains of health security is intended to function as a gateway to more nuanced case study analyses, in-country evaluations or after-action assessments that could help elucidate the causal mechanisms underpinning observed health outcomes during or after an outbreak. These latter approaches are better-suited to answering why, with respect to COVID-19, high-scoring countries like the USA, the UK, Sweden and Brazil have fared significantly worse than similarly high-ranked states like Thailand, and even compared with lower-ranked states like New Zealand, Vietnam and the Republic of Korea.

Such analyses of health system dynamics across specific country contexts are beyond the scope of the GHS Index itself, which solely monitors and documents the presence or absence of critical health security capacities. Therefore, to maximise the utility of the GHS Index, users should couple reviews of country scores with additional modes of data collection and analysis to answer questions regarding relationships between health capacities and health outcomes. Such approaches might include key informant interviews, surveys, in-country observations, case study analyses, focus groups or participatory action research methods.

Importantly, users should also note that the GHS Index is not a forecasting tool. Rather, it offers a cross-sectional snapshot of countries’ health security capacities as measured at one point in time. It cannot and does not predict whether or how a country will leverage available capacities to mitigate an unfolding crisis, nor does it make assumptions about the political decision-making processes underpinning these efforts. Furthermore, the inaugural GHS Index—which features data collected between 2018 and 2019 and was released in September 2019—does not account for many countries’ efforts to scale up testing and healthcare capacities in response to COVID-19, nor whether these efforts could be readily leveraged for future emergencies.

Finally, we reiterate that scores in all categories are intended to provide a comprehensive snapshot, based on publicly available data, of country capacities in these areas. Scores and their accompanying justifications are meant to inform—not supplant—country efforts to undertake JEEs and prioritise health security-strengthening measures in their respective NAPHS.

Next steps

Efforts are currently underway to further refine the GHS Index based on constructive feedback from experts and lessons learnt since the original model was developed. However, the GHS Index’s overarching finding still holds true: that no country is sufficiently prepared for epidemics or pandemics.1 The project team and International Panel of Experts fully anticipate that the GHS Index will undergo iterative refinement as stakeholders continue using the tool, identify shortcomings and learn more about COVID-19 and other diseases that may emerge in the future.

In light of the faults laid bare by COVID-19, we have already begun reexamining indicators of leadership, trust in government and political risk. The next iteration of the GHS Index, slated to release in 2021, will also emphasise other indicators that COVID-19 has shown to be vital for pandemic response, particularly those capturing state capacity to sustain societal functioning during a major outbreak. At present, we tentatively anticipate adding metrics of government effectiveness, as well as measures examining contact tracing and data sharing. We also anticipate revising several existing questions and adding over 20 new questions addressing zoonotic disease spillover events, scaling of testing capacities, abilities to test for novel pathogens, non-pharmaceutical interventions, private sector participation in exercises, risk communication, mis-information and disinformation, trade and travel, health security financing, social cohesion, illicit activities carried out by non-state actors, supply chains for medical supplies, laboratory supplies and medical countermeasures. Drawing further from the COVID-19 experience, we may also revisit weighting schemes for select categories and indicators.

In the longer term, we hope that the global health security community examines bigger-picture methodological issues around health security metrics. Many commonly used indicators of health security capacity—including those featured in the GHS Index, JEE and IHR Monitoring and Evaluation Framework—reflect a highly technocratic conceptualisation of outbreak prevention, detection and response. This framing likely fails to account for many social, institutional and cultural determinants of health system functioning and population health.

Furthermore, many of the indicators featured across all of these tools are investment-focused and capacity-focused. Future monitoring and evaluation efforts might benefit from deeper engagement with outcomes of health systems-strengthening and health security-strengthening efforts, such as health service coverage, health equity and health system failures. Subsequent iterations of the GHS Index specifically will have to contend with gaps in public data availability that disproportionately affect scores in LMICs. Inclusion of metrics capturing the quality and effectiveness of decision-making during a crisis might strengthen the tool even further.