The JEE process and core capacity grading

The JEE tool is a data gathering instrument designed to evaluate the IHR (2005) capacities in 19 technical areas, using 48 indicators categorised under four broad themes: prevent, detect, respond and other IHR hazards and PoE (table 1).38

The JEE tool and process has been widely accepted among the majority of WHO member states and national and international agencies because it was developed through international collaboration with Member States, subject matter experts, international organisations and existing initiatives. Scores for each of the 19 technical areas represent the arithmetic mean of the scores for the indicators of that technical area. Each indicator is scored on a 5-point ordinal scale. A score of 1 reflects no pertinent capacity, 2 is limited capacity, 3 is developed capacity, 4 connotes demonstrated capacity and a score of 5 reflects sustainable capacity (table 2).

The JEE is an evaluation and not research. It, therefore, does not require protocol approvals. Further, it is voluntary and emphasises a multisectoral approach for both the external team and the host country. It is an open, collaborative process for assessing a country’s IHR capabilities. It is a peer-to-peer evaluation and encourages transparency and openness of data and information sharing, and the public release of reports. Therefore, there are no ethical issues on the secondary analysis of JEE data as the JEE reports are available in the public domain.

Compilation, analysis and presentation of the JEE findings

Presently, there is no standardised weighting scheme to generate composite JEE indices from the 19 technical areas. Moreover, the benefit of presenting a single composite JEE index is also a subject for scientific debate. In view of the latter, we have elected to present the non-weighted arithmetic mean scores for each of the 19 technical areas and the number of countries within each score level. For visualisation purposes, we graphically present the data using a 5-category colour-coding to represent different score levels, specifically score 1=20%, no capacity (red), score 2=21%–40%, limited capacity (brown), score 3=41%–60%, developed capacity (yellow), score 4=61%–80%, demonstrated capacity (light green) and score 5=80% and above, sustainable capacity (dark green). While this analysis looks basic, it presents a non-biased baseline status of the IHR capacities in the WHO African region.

Current status of the International Health Regulation capacities in the WHO African region

Between, February 2016 and March 2019, 40 of 47 countries in the WHO African region (over 80% coverage) conducted a JEE (figure 1). A separate JEE was conducted for the island of Zanzibar. The WHO African region is the leading region, with over 42% (40 of 95) of the JEE conducted globally. The JEEs have demonstrated that no African country has all the required IHR capacities (figures 2 and 3).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Map of Africa showing countries that have completed the JEE (green), have JEE in the pipeline (yellow) and have not volunteered for a JEE (red). JEE, joint external evaluation.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

WHO African region JEE scorecard for the 19 JEE technical area (N=40), arithmetic means for prevent, detect, respond, PoE and other IHR hazards and for all 19 technical areas, AMR, antimicrobial resistance, IHR, International Health Regulations; JEE Joint External Evaluation; PH public health, PoE points of entry.

• Download figure
• Open in new tab
• Download powerpoint

Figure 3

The number of African countries in each IHR capacity score level for each of the 19 JEE technical areas (N=40), AMR, antimicrobial resistance, IHR, International Health Regulations; JEE Joint External Evaluation; PH public health, PoE points of entry.

In the broad thematic area of ‘prevent’, no country has capacity level 5 for 12 of 15 indicators. Capacity level 5 was recorded for three indicators: ‘surveillance systems for priority zoonotic diseases’ in one country, ‘vaccine coverage (measles) as part of the national programme in one country’ and ‘national vaccine access and delivery system in two countries’. For three technical areas (legislation, policy and financing, IHR coordination, communication and advocacy and antimicrobial resistance)>80% of the countries had either no capacity (score 1) or limited capacity (score 2). For two other technical areas (food safety and biosafety and biosecurity), >70% of the countries had either no capacity or limited capacity. The technical area with relatively higher IHR capacity scores was immunisation (figures 2 and 3). In the thematic area of ‘detect’, no country had capacity level 5 for 12 of the 13 indicators. Capacity level 5 was recorded for only one indicator ‘laboratory testing for priority diseases in one country’.

In contrast to prevent, many countries had a capacity level 3 or 4 for several indicators: 70% for ‘laboratory testing for detection of priority diseases’ and ‘efficient reporting to FAO, OIE and WHO’, 80% for ‘indicator or event-based surveillance systems’ and ‘field epidemiology training programmes’ and over 80% for ‘integration and data analysis’. Except for a few countries, workforce capacity had very low scores (figures 2 and 3). In the thematic area of ‘respond’, no country had capacity level 5 for 13 of 14 indicators. Less than 10% of the countries had a national multihazard public health emergency preparedness and response plan, and hardly 7% had mapped priority risks and resources. Over two-thirds of the countries had either no capacity or limited capacity to activate emergency operations and close to 80% of them had neither a public health emergency operation centre nor an emergency operations programme. Over 80% of them did not have linkages between public health and security authorities nor did they have mechanisms for receiving or sending medical personnel and medical countermeasures. The technical area in which one-third of the countries had either capacity level score 3 or 4 was risk communication. For the PoE and other IHR hazards (chemical events and radiation emergencies), no country had capacity level 5; a few had IHR capacity levels at score 3 or 4. Most countries (>80%) had an IHR capacity level at score 1 or 2 (figures 2 and 3). Finally, for illustration purposes, we present the arithmetic mean indices for the four major themes of the JEE tool (prevent, detect, respond, PoE and other IHR hazards) and the arithmetic mean for the 19 technical areas to demonstrate that a single JEE index, even without weighting, masks differences across the 19 technical areas (figure 3).

Putting the current IHR capacities status into context and perspectives for the future

Countries in the WHO African region are commended because over 80% of them have overwhelmingly embraced the JEE. Given the high burden of outbreaks and other public health emergencies, it is important that African countries use robust evidence to revise or develop their NAPHS.41 42 At the time of writing, of 40 countries that had completed a JEE, 24 had completed the revision/formulation and costing of their NAPHS, nine were in the process and five were planned. This rapid turnaround is attributed to learning lessons from the first country (ie, Tanzania) that conducted a JEE and subsequently revised their national action plan.43

There have been a few analyses of JEE data from the WHO African region.44 45 However, our paper is the most extensive documentation of the IHR (2005) capacities, covering over 80% of the countries.

Overall, these JEE findings are ‘a red flag’ about the inadequate public health emergency preparedness and response capacities in the WHO African region. This is a clarion call for the collective resolve of countries and their partners to build and sustain stronger national public health capabilities, infrastructure and processes as required under the IHR.33 Progress in IHR implementation should be monitored regularly and evaluated based on all components of the IHR monitoring and evaluation framework, including a repeat JEE every 4–5 years.37

It is clear that real-time surveillance and immunisation are the technical areas with the highest IHR capacity scores. This is not a random observation. These two programmes have been in place since the 1980s and the late 1990s for the expanded programme for immunisation and the IDSR, respectively. The IDSR strategy was adopted by the WHO African region in 1998 and has contributed to the strengthening of real-time surveillance (indicator and event-based) in human health.19–29 However, in several countries, major disparities were observed between the human and animal health sectors. Moving forward, there is a need to implement the ‘One Health Approach’.46

This analysis has revealed major gaps. First, most countries had not updated their public and animal health acts to take into consideration the broad scope of IHR (2005). Second, there was inadequate and untimely access to financial resources for the implementation of IHR (2005). Even in countries where financing was available, funds were often not accessed and distributed promptly. A third gap was the lack of resilient health systems.47 Resilient national health systems are essential for countries to prevent, detect, respond to and recover from PHEs and emergencies. Countries need to build health system resilience through the collective efforts of all relevant policymakers, stakeholders, practitioners and communities over a sustained period. A fourth major gap was inadequate multisectoral collaboration. Multisectoral and multidisciplinary coordination mechanisms are needed to facilitate efficient, alert and responsive systems for effective implementation of the IHR. This requires a functioning national IHR focal point (NFP) that is a national centre and not an individual.17 This is a key obligation of the IHR. All countries should strengthen their IHR NFP and should establish/strengthen multisectoral and multidisciplinary coordination mechanisms that integrate relevant sectors. A couple of countries are already establishing national public health institutes to play this role.

To address the above health security gaps, all countries in the WHO African region should urgently mobilise resources to support the implementation of their NAPHS. Addressing health security should be embedded into strategies for addressing universal health coverege.48 Establishing critical public health functions is a sovereign responsibility of countries, but the means to fulfil that responsibility are global. The IHR (2005) constitute the essential vehicle for that action. For long-term sustainability, funding for IHR implementation should be from domestic resources. The cost of conducting a single JEE in a country is approximately US$50 000. Therefore, the estimated cost of conducting JEEs in all 47 countries every 4–5 years in the WHO African region is US$2–3 million.

Based on cost estimates for pandemic preparedness from 24 national action plans for health security, we estimate, approximately US$ 9–10 billion is needed over the next 3 years for the whole of the WHO African region. This translates into US$ 2.50–3.50 per person per year—making the investment case for pandemic preparedness an affordable public health good. These investment needs are consistent with a 2016 global estimate of US$ 4.5 billion per year that was made by a US National Academy of Medicine commission on the global health risk framework for the future.33

Lessons learnt

The JEE is a new type of evaluation built on transparency and trust. Some of the lessons learnt during the conduct of these 41 JEEs are the following. The JEEs have been instrumental in bringing together several stakeholders from different sectors. In some countries, staff from the human and animal sectors were brought together for the first time because of the JEE. Further, security authorities were also brought on board for the first time because of the JEE.49 The JEE is helping to break down silos among human, plant, animal health, the environmental sectors and security and port authorities in ways not witnessed before. At the regional and global levels, the JEEs in the WHO African region are galvanising multiple stakeholders to work together on health security.

The JEE stimulates participatory, open debate both during the internal self-assessment and during the JEE. Because the JEE is a peer-to-peer assessment, it is essential that the JEE external team leadership ensures that all external subject matter experts especially those participating in a JEE for the first time understand this basic tenet of the JEE.

In some JEEs, participants tended to focus on a few shortcomings of the JEE tool. Like most assessment tools, the JEE tool is not perfect. The experience of the WHO African region using version 1 of the JEE tool was shared and utilised to revise the tool. Another common observation was stakeholders with ongoing projects in a given technical area tended to overscore themselves for accountability purposes, while those with projects that were winding up or projects that were not well funded tended to underscore themselves to lobby for funding. To limit these biases, it is essential for the external validation team to emphasise the need for openness and transparency in reflecting a country’s IHR capacity as truthfully as possible.

Challenges

Several challenges were observed: some related to the JEE process and others related to the tool. There are also challenges concerning the analysis and presentation of aggregate JEE indices. One of the challenges was getting the countries to conduct a JEE amidst other priorities. In some countries, there were inadequate human resources to conduct the self-assessment without external assistance.

Further, some national counterparts had inadequate understanding of the JEE process. In many instances, the JEE calendar had to be abruptly amended leading to setbacks in constituting multidisciplinary teams. However, this was eased as the rooster of JEE subject matter experts increased. Further, in some settings, instability and conflicts posed major challenges and necessitated innovative ways of conducting the validation field visits.

The JEE tool is global, and there have been situations where certain aspects of the assessment are not applicable to small states or are very challenging in countries with a federal system of government. A recent WHO technical meeting has provided recommendations on how to address these challenges without affecting the integrity of the tool.

With respect to analysis and presentation of the JEE findings, specifically, the computation of aggregate JEE indices, we believe standardisation and some form of weighting is required, depending on the hazard context. For instance, real-time surveillance has four indicators: zoonotic disease has three indicators and radiation emergencies has two indicators. Further, the hazard burden varies depending on the context. Therefore, aggregating technical area scores to generate a single or thematic JEE indices could benefit from a robust weighting scheme that is based on a ‘theory of change’ from experts in health security to avoid subjective and biased JEE indices. We plan to investigate further with experts how to standardise the computation of JEE indices. We will also explore the benefits of single or thematic aggregate JEE indices.