Discussion
There are multiple factors that create a challenging environment for fully understanding the impact of COVID-19 relative to existing external assessments. Some factors include consistent generation of high-quality data, availability of and competition for scarce resources such as PPE and vaccines, and imperfect understandings of variation between and within populations. For example, the availability of comparable data is a persistent challenge with international comparisons of COVID-19 outcomes. Detailed age-specific mortality rates are currently only available for 22 countries. COVID-19 case counts are further affected by variable country-specific testing capacity, inclusion criteria and reporting. Data on COVID-19 deaths are similarly limited and under-reported due to differences in vital statistics performance across the world. Additionally, the effect of intense competition for vaccines clearly suggests a strong influence of national wealth on COVID-19 outcomes. However, higher-income countries also tend to have older populations. Thus, examinations of excess mortality that are adjusted for age provides urgent information for assessing the role health security capacities have in mitigating COVID-19 burden.
This analysis therefore represents the first direct comparison of COVID-19 excess mortality rates across countries that accounts for under-reporting and national age structure. We found that after adjustment for income, higher GHS Index scores were associated with lower CMRs for excess COVID-19 mortality. The adjusted analysis confirms the expected relationship to preparedness illustrating that efforts to prepare for and respond to pandemics before they occur are effective in reducing mortality during global health emergencies. It is noteworthy that the relationship for the GHS Index remained present even after accounting for the strict thresholds during multiple hypothesis adjustments. While many of the individual capacities that were significant under the traditional thresholds did not meet the strict threshold established by the multiple hypothesis adjustment, there still remains evidence that associations are present. In fact, there is significant debate regarding the use of this correction, particularly in the context of this study where there are correlated and inter-dependent preparedness capacities, a limited sample size, and already predetermined hypotheses. The conservative multiple hypothesis adjustment also raises concerns of Type II error (concluding that a capacity is not significant when it truly is). Due to these concerns, we therefore are providing, for transparency, interpretations of findings that were present both prior and after conservative adjustments of multiple hypotheses throughout.
Our findings underscore that the core pandemic preparedness capacities of infectious disease prevention, detection and response are each associated with lower excess COVID-19 deaths. For example, prevention capacities may have reduced excess COVID-19 deaths by impeding the emergence of other infectious disease outbreaks35 36 that may have further burdened health systems and contributed to more mortality during the pandemic. In this context, our finding that the prevention indicator of immunisation capacities and rates being associated with fewer excess deaths may have appeared as this capacity likely minimised the number of vaccine preventable deaths37–39 and provided an infrastructure for successful COVID-19 vaccination programmes.40 41
We further observed that detection capacities, specifically capacities related to laboratory systems for detection of priority diseases and case-based investigations, were associated with less excess COVID-19 deaths. These findings are aligned with previous work illustrating that these capacities allow for early identification of cases,42 43 which increase the likelihood of early access to treatment, isolation of cases to minimise disease transmission and supports the effectiveness of mitigation strategies.44–46 These early detection capacities therefore contribute to improved health outcomes and fewer excess deaths.14 In addition, the finding that case-based investigation capacities were associated with reductions in excess deaths is consistent with previous work illustrating that these strategies reduced COVID-19 transmission47 48 and case fatality rates.49
Our findings also show that capacities for rapid responses to mitigate disease spread are associated with reduced COVID-19 burden.50–52 In particular, our results indicate that having a framework for emergency preparedness and response, which includes having health emergency plans, non-pharmacological intervention plans and considerations of vulnerable populations, was associated with fewer excess COVID-19 deaths. We may have observed this relationship owing to previous investigations finding that a lack of health emergency plans may lead to ineffective implementation of mitigation strategies.53–55 Therefore, having a framework for emergency response may equip countries with existing strategies that they can draw on during emergencies. Another response capacity that was related to reduced excess deaths was access to communication infrastructure. A myriad of studies has indicated that communication of disease risks increases knowledge of the disease56–58 and adherence to interventions,59 60 with some studies suggesting that risk communication is one of the most effective COVID-19 mitigation strategies.61 62 We may have found a strong relationship for communication infrastructure, as this capacity may have been essential for implementation of risk communication strategies in populations.
However, we did not observe an association between the health system category, a metric of health systems’ abilities to successfully treat patients, and excess deaths after adjustment for multiple hypotheses. Though we did not find an association, there was an effect prior to adjustment for multiple hypotheses. Further, there are numerous studies confirming that greater health system capacities are indeed associated with less COVID-19 burden by improving treatment outcomes,63–65 and that stronger health systems can minimise disruptions to essential services66 67 and therefore subsequently advert excess deaths. Our findings also show that capacity in healthcare settings, a core indicator of health system performance assessing available human resources and hospital beds in countries, was negatively related with excess COVID-19 deaths. Recent evidence indicates that settings with fewer human resources in healthcare settings are more vulnerable to excess COVID-19 deaths due to greater disruptions to essential health services.68 Therefore, considering that there was an effect without correction for multiple hypotheses and prior research, there is some evidence that investments in health systems can modulate pandemic outcomes. It is important to amass timely and accurate global data to more fully measure the strength and resilience of health systems to respond to infectious disease emergencies, while also meeting countries’ full set of health needs. Future studies should re-evaluate the role of health systems in supporting effective pandemic responses as global metrics of health system capacities improve.
Furthermore, we observed that other core GHS capacities, adherence to global norms and risk environment, not regularly assessed by other measures of pandemic preparedness were associated with diminished mortality. In regard to adherence to international norms, our findings provide some evidence that cross-border agreements are beneficial during a pandemic. For example, countries in the European Union shared the burden of the pandemic, as countries accepted hospitalised patients from overwhelmed countries, borders remained open to healthcare workers and those seeking medical care, and they shared essential knowledge.69 While these cross-border agreements have been shown to be difficult to implement due to differing country-specific rules and priorities,70 our results provide quantitative evidence that these collaborations can play a critical role in adverting deaths and major disruptions in care.
Finally, the GHS Index category that had the strongest and consistent relationship with excess COVID-19 mortality was the risk environment. The risk environment category assesses the socioeconomic, political, regulatory and ecological factors that increase vulnerability to outbreaks.71 A notable risk environment indicator that was associated with excess deaths was government effectiveness, which captures governments’ abilities to efficiently formulate and implement policies and accountability of public officials. This indicator was likely an important factor in cross-country variation of excess deaths as this capacity provides a framework for proactive policies to ensure supply of medical equipment and rapid implementation of interventions.72 73 We also found that levels of inequalities and social exclusion were each associated with fewer excess deaths. Across various countries investigations have highlighted that COVID-19 disproportionately affects vulnerable populations, as they are the least protected and often face the greatest risk from COVID-19.74–77 These discrepancies further propagate the pandemic and serve to exacerbate existing inequalities.78 Countries with lower levels of inequality were likely able to craft equitable responses that contributed to lower excess deaths and thus future preparedness plans should include measures to reduce disparities.79
The risk environment may be a primary reason why the US response was disjointed compared with other high-income countries. Despite the US ranking the highest in the GHS Index, the USA had the 41st smallest CMR and 30th largest risk environment score among the 57 high-income countries included in this analysis. While countries such as Iceland, Australia and New Zealand had the top 4 lowest CMRs and in the top 20 in risk environment. Evidence suggests that New Zealand was able to mount a success response because of strong leadership coordinating with many institutions to implement response measures in real-time, prioritisation of vulnerable populations in responses, effective communication strategies that induced population-wide support of responses and swift institutional approval of pandemic tools.80 81 Responses in Australia82 and Iceland83 also benefited from similarly strong, rapid and coordinated responses. While the USA has a multitude of pandemic capacities, the US response was fragmented due to states implementing different control strategies,84 early institutional rules preventing rapid mobilisation of diagnostic equipment85 and mixed communication that potentially harmed compliance in response measures.86
Overall, our analysis confirms that after adjustment for population age distribution and under-reporting of deaths, there are the expected country-level relationships between pandemic preparedness capacities and COVID-19 outcomes. Even after adjustment for GDP per capita as a confounder, owing to countries with greater income potentially having more resources to augment capacities and to allocate to health services to advert mortality, many capacities remained associated with reduced COVID-19 mortality. Our findings were also confirmed in our sensitivity analysis, where we further adjust for country-level differences in COVID-19 mitigation policies. These findings reinforce that regardless of income levels and real-time pandemic response policies, existing pandemic preparedness capacities, measured with the GHS Index, provide countries with a directly modifiable tool that they can build to avert mortality in the context of an evolving pandemic.
While our findings confirm the expected relationships between pandemic preparedness and COVID-19 outcomes, we identified a few capacities that were not associated with excess deaths. For example, previous studies have identified that greater levels of trust are associated with reduced COVID-19 burden,23 87–89 but we did not observe this relationship. However, we did find a relationship for public confidence in government, an analogous form of intuitional support and cooperation but confidence differs from trust in that it is built off previous evidence and experience.87 Thus, our analyses still provide some evidence that social and governmental support are important factors for responses to the pandemic. Future studies should continue to explore the country-level relationship between trust and COVID-19, and other capacities that were not related to excess deaths in this study including healthcare access and intervention planning.
Lastly, we found that the relationships between preparedness capacities and excess mortality became null when using data from the WHO and The Economist. A major contributor to the change in relationships is due to substantial differences in estimates between the three groups. For example, in countries with low GHS scores (<40), excess mortality estimates are generally twofold to threefold greater when comparing IHME to WHO estimates. Since these locations generally do not have reliable cause of death data, all three modelling groups rely on statistical models with various covariates and assumptions. Our initial investigations have shown that CMRs from the WHO and The Economist are moderately correlated with reported COVID-19 deaths while there is no correlation for CMRs produced from IHME estimates. Since under-reporting of COVID-19 deaths is a common problem in countries with low GHS scores, with postmortem surveillance studies in Africa indicating that deaths are undercounted by a factor of 10,90 91 the potential greater reliance on reported COVID-19 deaths by WHO and The Economist may partially explain the different estimates in countries with low GHS scores. Besides varying reliance on reported deaths, all three modelling groups also use different sets of covariates to produce estimates in locations without data. Overall, this sensitivity analysis revealed that pandemic preparedness capacities are not associated with worse pandemic outcomes and that there is a critical need for improved and robust pandemic outcome measures.
Strengths and limitations
This study has several strengths, including the ability, for the first time, to directly compare country-level COVID-19 excess mortality adjusted for age structure using CMRs. Our analysis was also able to evaluate multiple indicators of pandemic preparedness. This provides health systems with a collection of specific capacities that can be further evaluated to potentially modulate their vulnerability to the current pandemic and future global health emergencies. The identification of specific capacities is particularly timely as recent estimates show unprecedented increases in development assistance towards pandemic preparedness in low-income to middle-income countries.92
However, the results from this investigation should be interpreted in the context of the following limitations. First, our outcome data, excess COVID-19 deaths, are subject to measurement error due to varying levels of reliable capacities for vital registration systems and ability to enumerate all-cause mortality across countries. Due to a lack of data in Sub-Saharan Africa and Asia, the quantification of excess COVID-19 deaths in almost all countries in these regions was estimated using a statistical model with various predictive covariates.3 This is a limitation that is consistent for all three modelling groups of excess mortality. Though the excess mortality data used in this analysis are best estimates, the substantial lack of data in Sub-Saharan Africa and Asia reinforces the need to strengthen detection capacities in these areas. The lack of data based on direct measurement resulted in varying estimates of excess mortality by differing modelling groups, which was reflected in our analyses. The observation that excess death models that relied more heavily on countries’ reported COVID-19 deaths generated different results in our analysis, underscores the potential for heterogeneity in national surveillance capacities to affect our ability to track deaths at the global level. The inability to fully enumerate disease-specific mortality is a critical gap in global pandemic surveillance. Efforts to improve national surveillance for infectious disease emergencies must also include efforts to bolster countries’ vital registration and all-cause mortality surveillance. In the interim, countries may consider improving their surveillance by employing survey methods such as postmortem surveillance studies. One such surveillance study in Zambia found that actual COVID-19 deaths are 10 times greater than reported deaths.90 These methods may assist in constructing more robust measures of COVID-19 impact and therefore assist future studies in providing more robust evaluations of the contributions of pandemic preparedness capacities.
Second, a similar limitation is that due to the lack of age-specific data on COVID-19 mortality in Sub-Saharan Africa and Asia, we were not able to conduct sensitivity analyses using countries from these regions as the reference in computations of CMRs. Some evidence suggests that the age pattern of COVID-19 mortality is steeper in the elderly age groups in high-income countries while flatter in non-high income,18 differences that may potentially yield differing distributions of CMRs. Third, there is potential measurement error in the GHS Index as the metric was constructed using data that was publicly available and therefore may not capture capacities that are not written up or published. Similarly, the country-level analyses may obscure important variation in pandemic preparedness capacities within countries as capacities may substantially vary within countries. Third, the GHS Index–COVID-19 relationship is likely to change as the pandemic progresses because the outcome is still developing with new reliable data becoming available. Fourth, our analytic approach assumed a linear relationship between the GHS Index measures and COVID-19 excess mortality. Fifth, the multiple hypothesis adjusted analysis was conservative and may have introduced Type II error. We therefore presented results prior and after adjustments for multiple hypotheses. Finally, the ecological nature of the data prevents us from making inferences regarding pandemic preparedness capacities and excess mortality at the individual-level.