Article Text

Joint external evaluation of the International Health Regulation (2005) capacities: current status and lessons learnt in the WHO African region
1. Ambrose Talisuna1,
2. Ali Ahmed Yahaya1,
3. Soatiana Cathycia Rajatonirina1,
4. Mary Stephen1,
5. Antonio Oke1,
6. Allan Mpairwe1,
7. Amadou Bailo Diallo1,
8. Emmanuel Onuche Musa1,
9. Daniel Yota1,
10. Freddy Mutoka Banza1,
11. Roland Kimbi Wango1,
12. Nathalie Amy Roberts2,
13. Rajesh Sreedharan2,
14. Nirmal Kandel2,
15. Adrienne May Rashford2,
16. Linda Lucy Boulanger2,
17. Qudsia Huda2,
18. Stella Chungong2,
19. Zabulon Yoti1,
20. Ibrahima Soce Fall1
1. 1WHO Health Emergency Programme, World Health Organization, Regional Office for Africa, Brazzaville, Republic of Congo
2. 2WHO Health Emergency Programme, World Health Organisation, Geneva, Switzerland
1. Correspondence to Ambrose Talisuna; talisunaa{at}who.int

## Abstract

The International Health Regulations (IHR, 2005) are an essential vehicle for addressing global health security. Here, we report the IHR capacities in the WHO African from independent joint external evaluation (JEE). The JEE is a voluntary component of the IHR monitoring and evaluation framework. It evaluates IHR capacities in 19 technical areas in four broad themes: ‘Prevent’ (7 technical areas, 15 indicators); ‘Detect’ (4 technical areas, 13 indicators); ‘Respond’ (5 technical areas, 14 indicators), points of entry (PoE) and other IHR hazards (chemical and radiation) (3 technical areas, 6 indicators). The IHR capacity scores are graded from level 1 (no capacity) to level 5 (sustainable capacity). From February 2016 to March 2019, 40 of 47 WHO African region countries (81% coverage) evaluated their IHR capacities using the JEE tool. No country had the required IHR capacities. Under the theme ‘Prevent’, no country scored level 5 for 12 of 15 indicators. Over 80% of them scored level 1 or 2 for most indicators. For ‘Detect’, none scored level 5 for 12 of 13 indicators. However, many scored level 3 or 4 for several indicators. For ‘Respond’, none scored level 5 for 13 of 14 indicators, and less than 10% had a national multihazard public health emergency preparedness and response plan. For PoE and other IHR hazards, most countries scored level 1 or 2 and none scored level 5. Countries in the WHO African region are commended for embracing the JEE to assess their IHR capacities. However, major gaps have been identified. Urgent collective action is needed now to protect the WHO African region from health security threats.

• Africa
• acute public health event
• health security
• health emergencies
• integrated disease surveillance and response
• international health regulations
• outbreaks
• world health organisation

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

## Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

### Summary box

• The International Health Regulations (IHR, 2005) are an essential vehicle for addressing global health security. Previously, countries have been self-reporting their IHR capacities. Here, we report the IHR capacities in the WHO African region from independent joint external evaluation.

• No country had all the required IHR capacities. The immunisation technical area was a common strength for many countries. No country had ‘no capacity’ (level 1), and seven countries (18%) had demonstrated (level 4) or sustainable (level 5) capacity. Similarly, no country had ‘no capacity’ (level 1) for real-time surveillance and one country had sustainable (level 5) capacity.

• Major gaps were observed in the following technical areas: antimicrobial resistance, biosafety and biosecurity, preparedness, emergency response operations, medical countermeasures and personnel deployment, PoE, chemical events and radiation emergencies.

• Moving forward, decisive action is needed now to ensure that people in the WHO African region are better protected from epidemics and other public health emergencies. All countries in the WHO African region should urgently establish robust national public health capabilities, infrastructure and processes. The latter should be assessed regularly through an objective and transparent process.

## Introduction

Outbreaks and other acute public health emergencies continue to affect the WHO African region. An acute public health event (PHE) is reported every 3–4 days, which is more than 150 acute PHEs annually.1 The entire WHO African region is at risk of health security threats.1 2 Of particular concern are emerging and re-emerging pathogens. For instance, the Ebola and Marburg Virus Disease outbreaks, which were previously rare, have recently caused devastating outbreaks in the region.3–12 The top three causes of infectious disease outbreaks in 2017 were cholera, viral haemorrhagic diseases and measles.1 Further, several outbreaks of meningococcal meningitis have recently occurred outside the meningitis belt, suggesting that the latter may be expanding.13 Moreover, humanitarian crises continue to disrupt livelihoods, and the economy of the countries at risk.14 Further, rapid population growth, unplanned rapid urbanisation and the effects of climate change continue to impact negatively on the region.15 16

The IHR (2005) constitute the essential vehicle for addressing global health security.17 The IHR (2005) aim at protecting global health security while avoiding unnecessary interference with international traffic and trade.17 Under the IHR, countries are obliged to develop and maintain the required capacities for surveillance and response, to detect, assess, notify and respond to any public health emergency of potential international concern.17 In accordance with the IHR, countries must report their IHR implementation status annually to the World Health Assembly (WHA) and the WHO Executive Board.17 18

In the WHO African region, the implementation of the IHR has previously been facilitated by the implementation of the integrated disease surveillance and response (IDSR) and the disaster risk management strategies.19–29 However, the Ebola virus disease outbreak of 2013–2016 in West Africa highlighted major gaps in IHR implementation.30–33 According to the self-assessment reports, by 2016, no country in the WHO African region had all the required IHR capacities.34 The latter is attributed to inadequate health systems in most countries.35

Before 2015, countries were self-reporting their IHR implementation status, annually to the WHA.17 However, several IHR review committees and various experts’ panels have recommended, in addition to mandatory annual reporting, three voluntary components. These include After Action Reviews, Simulations and Exercises and importantly, voluntary joint external evaluation (JEE).33 36 37 Consequently, in 2015, WHO and partners developed the JEE tool based on existing tools,38 such as the WHO IHR self-assessment questionnaire,26 the Global Health Security Agenda assessment tool,39 and the Organization for Animal Health (OIE) Performance of Veterinary Services pathway tool.40 The JEE among others provides an objective basis for the evidence-led formulation of national action plans for health security (NAPHS).37 41

Here, we present the baseline status of the IHR (2005) capacities in the WHO African region generated from the JEEs conducted between February 2016 and March 2019. Further, we share the challenges, best practices and lessons learnt. We believe that the experiences and lessons learnt from the WHO African region could motivate other WHO regions that are yet to embrace and scale up the JEEs, as part of evidence-led NAPHS.

## The JEE process and core capacity grading

The JEE tool is a data gathering instrument designed to evaluate the IHR (2005) capacities in 19 technical areas, using 48 indicators categorised under four broad themes: prevent, detect, respond and other IHR hazards and PoE (table 1).38

Table 1

JEE technical areas and number of indicators

The JEE tool and process has been widely accepted among the majority of WHO member states and national and international agencies because it was developed through international collaboration with Member States, subject matter experts, international organisations and existing initiatives. Scores for each of the 19 technical areas represent the arithmetic mean of the scores for the indicators of that technical area. Each indicator is scored on a 5-point ordinal scale. A score of 1 reflects no pertinent capacity, 2 is limited capacity, 3 is developed capacity, 4 connotes demonstrated capacity and a score of 5 reflects sustainable capacity (table 2).

Table 2

The JEE grading and scoring

The JEE is an evaluation and not research. It, therefore, does not require protocol approvals. Further, it is voluntary and emphasises a multisectoral approach for both the external team and the host country. It is an open, collaborative process for assessing a country’s IHR capabilities. It is a peer-to-peer evaluation and encourages transparency and openness of data and information sharing, and the public release of reports. Therefore, there are no ethical issues on the secondary analysis of JEE data as the JEE reports are available in the public domain.

## Compilation, analysis and presentation of the JEE findings

Presently, there is no standardised weighting scheme to generate composite JEE indices from the 19 technical areas. Moreover, the benefit of presenting a single composite JEE index is also a subject for scientific debate. In view of the latter, we have elected to present the non-weighted arithmetic mean scores for each of the 19 technical areas and the number of countries within each score level. For visualisation purposes, we graphically present the data using a 5-category colour-coding to represent different score levels, specifically score 1=20%, no capacity (red), score 2=21%–40%, limited capacity (brown), score 3=41%–60%, developed capacity (yellow), score 4=61%–80%, demonstrated capacity (light green) and score 5=80% and above, sustainable capacity (dark green). While this analysis looks basic, it presents a non-biased baseline status of the IHR capacities in the WHO African region.

## Current status of the International Health Regulation capacities in the WHO African region

Between, February 2016 and March 2019, 40 of 47 countries in the WHO African region (over 80% coverage) conducted a JEE (figure 1). A separate JEE was conducted for the island of Zanzibar. The WHO African region is the leading region, with over 42% (40 of 95) of the JEE conducted globally. The JEEs have demonstrated that no African country has all the required IHR capacities (figures 2 and 3).

Figure 1

Map of Africa showing countries that have completed the JEE (green), have JEE in the pipeline (yellow) and have not volunteered for a JEE (red). JEE, joint external evaluation.

Figure 2

WHO African region JEE scorecard for the 19 JEE technical area (N=40), arithmetic means for prevent, detect, respond, PoE and other IHR hazards and for all 19 technical areas, AMR, antimicrobial resistance, IHR, International Health Regulations; JEE Joint External Evaluation; PH public health, PoE points of entry.

Figure 3

The number of African countries in each IHR capacity score level for each of the 19 JEE technical areas (N=40), AMR, antimicrobial resistance, IHR, International Health Regulations; JEE Joint External Evaluation; PH public health, PoE points of entry.

In the broad thematic area of ‘prevent’, no country has capacity level 5 for 12 of 15 indicators. Capacity level 5 was recorded for three indicators: ‘surveillance systems for priority zoonotic diseases’ in one country, ‘vaccine coverage (measles) as part of the national programme in one country’ and ‘national vaccine access and delivery system in two countries’. For three technical areas (legislation, policy and financing, IHR coordination, communication and advocacy and antimicrobial resistance)>80% of the countries had either no capacity (score 1) or limited capacity (score 2). For two other technical areas (food safety and biosafety and biosecurity), >70% of the countries had either no capacity or limited capacity. The technical area with relatively higher IHR capacity scores was immunisation (figures 2 and 3). In the thematic area of ‘detect’, no country had capacity level 5 for 12 of the 13 indicators. Capacity level 5 was recorded for only one indicator ‘laboratory testing for priority diseases in one country’.

## Putting the current IHR capacities status into context and perspectives for the future

Countries in the WHO African region are commended because over 80% of them have overwhelmingly embraced the JEE. Given the high burden of outbreaks and other public health emergencies, it is important that African countries use robust evidence to revise or develop their NAPHS.41 42 At the time of writing, of 40 countries that had completed a JEE, 24 had completed the revision/formulation and costing of their NAPHS, nine were in the process and five were planned. This rapid turnaround is attributed to learning lessons from the first country (ie, Tanzania) that conducted a JEE and subsequently revised their national action plan.43

There have been a few analyses of JEE data from the WHO African region.44 45 However, our paper is the most extensive documentation of the IHR (2005) capacities, covering over 80% of the countries.

Overall, these JEE findings are ‘a red flag’ about the inadequate public health emergency preparedness and response capacities in the WHO African region. This is a clarion call for the collective resolve of countries and their partners to build and sustain stronger national public health capabilities, infrastructure and processes as required under the IHR.33 Progress in IHR implementation should be monitored regularly and evaluated based on all components of the IHR monitoring and evaluation framework, including a repeat JEE every 4–5 years.37

It is clear that real-time surveillance and immunisation are the technical areas with the highest IHR capacity scores. This is not a random observation. These two programmes have been in place since the 1980s and the late 1990s for the expanded programme for immunisation and the IDSR, respectively. The IDSR strategy was adopted by the WHO African region in 1998 and has contributed to the strengthening of real-time surveillance (indicator and event-based) in human health.19–29 However, in several countries, major disparities were observed between the human and animal health sectors. Moving forward, there is a need to implement the ‘One Health Approach’.46

This analysis has revealed major gaps. First, most countries had not updated their public and animal health acts to take into consideration the broad scope of IHR (2005). Second, there was inadequate and untimely access to financial resources for the implementation of IHR (2005). Even in countries where financing was available, funds were often not accessed and distributed promptly. A third gap was the lack of resilient health systems.47 Resilient national health systems are essential for countries to prevent, detect, respond to and recover from PHEs and emergencies. Countries need to build health system resilience through the collective efforts of all relevant policymakers, stakeholders, practitioners and communities over a sustained period. A fourth major gap was inadequate multisectoral collaboration. Multisectoral and multidisciplinary coordination mechanisms are needed to facilitate efficient, alert and responsive systems for effective implementation of the IHR. This requires a functioning national IHR focal point (NFP) that is a national centre and not an individual.17 This is a key obligation of the IHR. All countries should strengthen their IHR NFP and should establish/strengthen multisectoral and multidisciplinary coordination mechanisms that integrate relevant sectors. A couple of countries are already establishing national public health institutes to play this role.

To address the above health security gaps, all countries in the WHO African region should urgently mobilise resources to support the implementation of their NAPHS. Addressing health security should be embedded into strategies for addressing universal health coverege.48 Establishing critical public health functions is a sovereign responsibility of countries, but the means to fulfil that responsibility are global. The IHR (2005) constitute the essential vehicle for that action. For long-term sustainability, funding for IHR implementation should be from domestic resources. The cost of conducting a single JEE in a country is approximately US$50 000. Therefore, the estimated cost of conducting JEEs in all 47 countries every 4–5 years in the WHO African region is US$2–3 million.

Based on cost estimates for pandemic preparedness from 24 national action plans for health security, we estimate, approximately US$9–10 billion is needed over the next 3 years for the whole of the WHO African region. This translates into US$ 2.50–3.50 per person per year—making the investment case for pandemic preparedness an affordable public health good. These investment needs are consistent with a 2016 global estimate of US\$ 4.5 billion per year that was made by a US National Academy of Medicine commission on the global health risk framework for the future.33

## Lessons learnt

The JEE is a new type of evaluation built on transparency and trust. Some of the lessons learnt during the conduct of these 41 JEEs are the following. The JEEs have been instrumental in bringing together several stakeholders from different sectors. In some countries, staff from the human and animal sectors were brought together for the first time because of the JEE. Further, security authorities were also brought on board for the first time because of the JEE.49 The JEE is helping to break down silos among human, plant, animal health, the environmental sectors and security and port authorities in ways not witnessed before. At the regional and global levels, the JEEs in the WHO African region are galvanising multiple stakeholders to work together on health security.

The JEE stimulates participatory, open debate both during the internal self-assessment and during the JEE. Because the JEE is a peer-to-peer assessment, it is essential that the JEE external team leadership ensures that all external subject matter experts especially those participating in a JEE for the first time understand this basic tenet of the JEE.

In some JEEs, participants tended to focus on a few shortcomings of the JEE tool. Like most assessment tools, the JEE tool is not perfect. The experience of the WHO African region using version 1 of the JEE tool was shared and utilised to revise the tool. Another common observation was stakeholders with ongoing projects in a given technical area tended to overscore themselves for accountability purposes, while those with projects that were winding up or projects that were not well funded tended to underscore themselves to lobby for funding. To limit these biases, it is essential for the external validation team to emphasise the need for openness and transparency in reflecting a country’s IHR capacity as truthfully as possible.

## Challenges

Several challenges were observed: some related to the JEE process and others related to the tool. There are also challenges concerning the analysis and presentation of aggregate JEE indices. One of the challenges was getting the countries to conduct a JEE amidst other priorities. In some countries, there were inadequate human resources to conduct the self-assessment without external assistance.

Further, some national counterparts had inadequate understanding of the JEE process. In many instances, the JEE calendar had to be abruptly amended leading to setbacks in constituting multidisciplinary teams. However, this was eased as the rooster of JEE subject matter experts increased. Further, in some settings, instability and conflicts posed major challenges and necessitated innovative ways of conducting the validation field visits.

The JEE tool is global, and there have been situations where certain aspects of the assessment are not applicable to small states or are very challenging in countries with a federal system of government. A recent WHO technical meeting has provided recommendations on how to address these challenges without affecting the integrity of the tool.

With respect to analysis and presentation of the JEE findings, specifically, the computation of aggregate JEE indices, we believe standardisation and some form of weighting is required, depending on the hazard context. For instance, real-time surveillance has four indicators: zoonotic disease has three indicators and radiation emergencies has two indicators. Further, the hazard burden varies depending on the context. Therefore, aggregating technical area scores to generate a single or thematic JEE indices could benefit from a robust weighting scheme that is based on a ‘theory of change’ from experts in health security to avoid subjective and biased JEE indices. We plan to investigate further with experts how to standardise the computation of JEE indices. We will also explore the benefits of single or thematic aggregate JEE indices.

## Conclusions

Countries in the WHO African region have shown high-level commitment to assess their IHR capacities. However, major gaps have been observed. Moving forward, all countries should harness this coalition of partners and tap into the current momentum to develop cost and implement national action plans for health security to address the gaps identified. Such plans should be implemented in synergy and alignment with broader health systems strengthening strategies. This requires the highest level of advocacy and political commitment to mobilise adequate financing. Collective action is urgently needed now to protect the people in the WHO African region from epidemics and other public health emergencies.

## Acknowledgments

We would like to acknowledge the active participation of national experts from all the volunteering countries, the members of the international roster of experts-over 1000 of them, and the invaluable partnerships with governments including the governments of Finland, Germany and the United States; with other Intergovernmental organisations, particularly the United Nations Food and Agriculture Organization (FAO), the World Organisation for Animal Health (OIE) and the International Civil Aviation Organization (ICAO); many public health institutions such as the United States Centers for Disease Prevention and Control (US CDC), the Africa Centres for Disease Prevention and Control (Africa CDC), Resolve to Save Lives, the European Centres for Disease Prevention and Control (ECDC), the Japanese International Cooperation Agency (JICA), the China Centers for Disease Control and Prevention (China CDC), the World Bank and Public Health England (PHE); private entities such as the Bill and Melinda Gates Foundation and many other partners, including the members of the Global Health Security Agenda and of the joint external evaluation (JEE) Alliance. We would like to acknowledge the continuing support and commitment of all of these to the implementation and principles of the International Health Regulations (2005). Finally, we thank the Member States for their ownership and leadership in volunteering to conduct the JEE.

1. 1.
2. 2.
3. 3.
4. 4.
5. 5.
6. 6.
7. 7.
8. 8.
9. 9.
10. 10.
11. 11.
12. 12.
13. 13.
14. 14.
15. 15.
16. 16.
17. 17.
18. 18.
19. 19.
20. 20.
21. 21.
22. 22.
23. 23.
24. 24.
25. 25.
26. 26.
27. 27.
28. 28.
29. 29.
30. 30.
31. 31.
32. 32.
33. 33.
34. 34.
35. 35.
36. 36.
37. 37.
38. 38.
39. 39.
40. 40.
41. 41.
42. 42.
43. 43.
44. 44.
45. 45.
46. 46.
47. 47.
48. 48.
49. 49.

## Footnotes

• Handling editor Seye Abimbola

• Contributors AT, AAY, SCR, MS, AO, AM, ABD, EOM, DY, FMB, RKW, NAR, RS, NK, AMR, LLB, QH, SC, ZY, ISF planned, coordinated or participated in several JEEs. AT, RKW and NK analysed that data, AT wrote the first draft of the manuscript. All the authors reviewed and approved the final version.

• Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

• Competing interests None declared.

• Patient consent for publication Not required.

• Provenance and peer review Not commissioned; externally peer reviewed.

• Data sharing statement All data are available with WHO and reports are available online.