Negotiations are underway at the WHO for a legally binding instrument for pandemic prevention, preparedness and response. As seen in the International Health Regulations, however, countries signing up to an agreement is no guarantee of its effective implementation. We, therefore, investigated the potential design features of an accountability framework for the proposed pandemic agreement that could promote countries’ compliance with it. We reviewed the governance of a number of international institutions and conducted over 40 interviews with stakeholders and experts to investigate how the pandemic agreement could be governed.
We found that enforcement mechanisms are a key feature for promoting the compliance of countries with the obligations they sign up for under international agreements but that they are inconsistently applied. It is difficult to design enforcement mechanisms that successfully avoid inflicting unintended harm and, so, we found that enforcement mechanisms generally rely on soft political levers rather than hard legal ones to promote compliance. Identifying reliable information on states’ behaviour with regard to their legal obligations requires using a diverse range of information, including civil society and intergovernmental organisations, and maintaining legal, financial, and political independence.
We, therefore, propose that there should be an independent mechanism to monitor states’ compliance with and reporting on the pandemic agreement. It would mainly triangulate a diverse range of pre-existing information and have the authority to receive confidential reports and seek further information from states. It would report to a high-level political body to promote compliance with the pandemic agreement.
- health policies and all other topics
- health policy
Data availability statement
All data relevant to the study are included in the article or uploaded as supplementary information.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.
Accountability mechanisms are used through a variety of methods across global treaties and governance mechanisms to varying degrees of effectiveness.
The pandemic agreement should have accountability mechanisms built into it from the start to increase the likelihood of countries complying with the obligations they sign up for.
Accountability mechanisms require reliable information flowing about the performance of state parties with respect to their obligations under the agreement.
A monitoring mechanism, reviewing countries’ progress and their reporting, should be politically, financially, technically and operationally independent of the WHO and donors to increase the likelihood of its reliability.
The COVID-19 pandemic exposed deficiencies in the current structures for global health governance. The International Health Regulations (IHR), despite being legally binding on all WHO member states, did not lead to a coordinated and timely response. Many states lacked the capacity and/or the political will to undertake the necessary action to prepare for health emergencies and/or to respond after the WHO declared COVID-19 a Public Health Emergency of International Concern (PHEIC).1 The COVID-19 response in many countries even countered what the IHR stipulated and the WHO advised and was instead driven to a much larger extent by political interests.2
In response to these challenges in pandemic prevention, preparedness and response (PPR), a special session of the World Health Assembly (WHA) in November 2021 established an Intergovernmental Negotiating Body (INB) and tasked it with drafting a new legal instrument for PPR (henceforth ‘the pandemic agreement’). The INB is scheduled to submit a final draft to the WHA in May 2024. Member States are meeting simultaneously through the Working Group on Amendments to the IHR (2005) to negotiate over 300 proposed amendments to the IHR, also with a deadline of May 2024 to submit a consensus set of amendments. However, as seen with the IHR and many other treaties and instruments globally, merely adopting legally binding provisions does not guarantee countries’ compliance with the obligations they sign up for.3 4 The latest draft of the agreement therefore puts forward an ‘Implementation and Compliance Committee’, made up of experts nominated by State Parties, that would promote compliance with the treaty based on information received from state parties and other, unspecified, sources. This committee would be a subsidiary body of a ‘Conference of the Parties’ (COP) that would cooperate with the WHA.5
In order to inform the design of a compliance mechanism with the pandemic agreement, we study the potential features of an accountability framework, with the aim of proposing a design for a monitoring structure. We do this by identifying factors that influence compliance with other international treaties.
The most comprehensive systematic review to date on treaty monitoring found that enforcement mechanisms are the only design feature in global treaties that may increase the chance of their effective implementation.6 Besides enforcement mechanisms, the review found that extralegal factors, such as the socialisation of treaties and the political will around them, also contributed positively to their effective implementation.
Building on this and other work, we focus in particular on the monitoring component of an accountability mechanism that could maximise the chances of obtaining reliable information about countries’ performance with respect to their obligations under the agreement.
We conducted a literature review and 42 semistructured interviews for this analysis. We reviewed relevant peer-reviewed and grey literature that describes and analyses the governance of the Framework Convention on Tobacco Control (FCTC). The FCTC was chosen as it is the only treaty negotiated through the WHO to date. Because experience with treaties in the health sector is so limited, we also included in the review monitoring mechanisms of treaties and agreements outside of health, selected based on the recommendation by the interviewed experts.
The interviews were used to complement learnings and provide feedback on the findings from the literature review, and to review the relevance of those findings to design a system for independent monitoring of the pandemic agreement.
We selected the interviewees on the basis of their expertise on global health governance, health security and the treaties reviewed. We included a range of respondents from diverse geographies including academics, government officials, officials from international organisations and foundations, advocates, and activists.
Interviews were conducted on-line between December 2021 and September 2022, and lasted from 30 to 45 min, with two members of the research team present. In line with our institutional protocol, we obtained verbal consent confirming that participation was voluntary and anonymous. To maintain the comfort of the participants, we did not record interviews. We used Excel to organise and analyse the content of the obtained data.
As we did not record interviews and therefore did not take transcripts, rather than use direct quotes in this paper, we synthesise the ideas presented by experts.
The new data obtained through the systematic review and interviews builds on the existing findings of an October 2021 report by the research team, in which we mapped ten bodies and organisations monitoring progress towards strategies or compliance with international agreements.3 This is referenced where relevant.
The analysis synthesises our key findings. Following the Standards for Reporting Qualitative Research, it is structured as follows: in the sections ‘General Design Features of Monitoring Mechanisms’ and ‘Strengths and Weaknesses of Monitoring Mechanisms’, we present the findings obtained through a literature review and interviews. The section entitled ‘Proposal for an Independent Monitoring Committee’ presents our interpretation of these findings in the context of the pandemic agreement. We close with a discussion of the strengths and limitations of this paper and a conclusion.
General design features of monitoring mechanisms
We chose to review the FCTC because it is the only treaty adopted under Article 19 of the WHO Constitution to date. It, therefore, represents the only time the WHA previously used the process currently underway for the pandemic agreement. We also considered other monitoring mechanisms because there is limited experience with such conventions in the global health sector.
Governance of the FCTC control
The FCTC was adopted in 2003 and entered into force in 2005, and aims to strengthen multilateral cooperation and national action to tackle the global tobacco epidemic.7 It is governed by a COP, a meeting of state parties and observers to the convention that takes place every 2 years to monitor progress and take decisions to promote the convention’s effective implementation. Delegations generally consist of representatives from permanent missions to Geneva, national health officials, and officials from relevant non-health ministries such as trade and finance.8 Observers (non-state parties, intergovernmental organisations and non-governmental organisations) attend as non-voting members.9
Compliance monitoring with the FCTC mainly takes place through parties’ obligation to submit reports on their own progress to the COP, detailing their actions to comply with the Convention.10 Every 2 years at a minimum, states must use a digital ‘core questionnaire’ that the COP adopted in 2010 to consolidate the reporting mechanism. This is completed by technical focal points and local staff, who are appointed internally by countries. The reports cover each of the measures stipulated by the FCTC articles.11 The secretariat compiles these reports into a database through which they review progress of each state and collective implementation of each FCTC implementation area and publishes biennial global progress reports.12 13
The secretariat, which reports to the COP but is hosted by the Director-General’s Office at the WHO, works closely with relevant WHO departments but has programmatic independence on treaty matters. It is funded by state parties to the FCTC.12 14
Other monitoring mechanisms
In addition to analysing the governance of the FCTC, we considered ten other monitoring mechanisms. These included the COP of the United Nations Framework Convention on Climate Change (UNFCCC), the International Labour Organization (ILO) confidential reporting mechanism, the Independent Accountability Panel for the Every Woman Every Child strategy, the Financial Stability Board (FSB), the Intergovernmental Panel on Climate Change, the Independent Monitoring Board (IMB) for Polio, the Human Rights Treaty bodies, the Human Rights Council (HRC), the International Atomic Energy Agency (IAEA) Safeguards and the International Monetary Fund (IMF) Article IV mechanism. A summary of the purpose, composition, method of working and funding of each body is outlined in table 1.
Strengths and weaknesses of monitoring mechanisms
The literature review revealed that most of the mechanisms reviewed do not have clear enforcement mechanisms to promote compliance or even to guarantee accurate reporting by countries, relying instead, for the most part, on political pressure. In the case of the FCTC, for example, there are no direct methods for enforcing compliance or verifying reports. Civil society organisations (CSOs) who are observers to the COP, such as the Framework Convention Alliance (FCA), can play the role of verifying reports but this is not formalised in the treaty structures; it is dependent on the priorities and capacity of the observers.15 16 Reviews of the COP have urged the adoption of a more robust monitoring system.17 18 However, several COP sessions considered but failed to adopt an implementation review framework to promote accountability.19 Several experts highlighted the significant disparities in CSO engagement in the FCTC because the treaty lacks a formal shadow-reporting system, which in turn means the effectiveness of CSOs is dependent on how well-resourced they are.
The literature review also showed how treaties try to use political levers to promote compliance more directly, using technical support and reputational incentives. The UNFCCC COP, for example, uses a ‘facilitative’ approach; this approach assumes that state parties fail to comply because they lack capacity to do so. The secretariat is therefore mandated to provide technical support and resources to states that fall short of their obligations.20 The Human Rights Treaty Bodies and the ILO, on the other hand, rely on reputational incentives (‘naming and shaming’), where countries are called out through public reports when they are non-compliant with human rights obligations or labour regulations, respectively.21 22 These are sometimes linked to repeat assessments to assess the extent to which states have rectified non-compliance.22
We identified two examples of mechanisms with clear enforcement frameworks: the IMF Article IV Mechanism and the IAEA. Non-compliance with IMF Article 4 may result in indirect financial sanctions, in the form of IMF loans being denied or lower ratings from the IMF, which may negatively impact financial investments in the country in question.23 24 Non-compliance with the IAEA Safeguards may be escalated to the UN General Assembly (UNGA) and/or the Security Council.3 They in turn may issue sanctions. Several experts noted that sanctions are rarely helpful; they have been shown to harm populations with limited evidence that they change countries’ behaviour.
Experts generally agreed that the IHR’s effectiveness is limited by lack of compliance with it. They broadly agreed, in principle, with the importance of establishing enforcement mechanisms for countries’ obligations with the pandemic agreement to address the shortcomings of the IHR. They raised concerns, however, that it may not be feasible to design enforcement mechanisms that avoid inadvertent harm, especially through sanctions, which generally harm the most vulnerable in society. Many experts were also concerned that even positive incentives, such as tying access to technologies and products with compliance, may carry the same risk of preventing access to essential resources for the most vulnerable in society. This left reputational incentives, through political pressure from other states and civil society, as means to promote compliance as the option that experts were least resistant to. There are significant disparities, however, in the capacity of CSOs to hold their countries to account, which is dependent on their respective resources. Some experts suggested that countries’ poor reporting and lack of compliance are usually a result of resource constraints, rather than lack of willingness. This led them to support the principles adopted in the climate change treaty regime that technical support and resources are the most promising solutions.
The literature revealed that the independence of accountability structures vary. The HRC, for example, relies on representative bodies to monitor compliance. However, this has been criticised as prone to political interference.25 Other mechanisms such as the IAEA, IMB and UNFCCC, thus emphasise independence.3
Among experts, there was general agreement with the principle of independence over peer review, because peer review may be influenced by political considerations. Several, however, cautioned about how this could be realistically implemented and noted that independence could be threatened by funding and the influence of donors, institutional arrangements, and cultural factors such as ‘groupthink’. Experts also pointed out that countries may reject independent monitoring if they perceive that it may infringe on political sovereignty.
Sources of information
The October 2021 report by the research team found that different bodies vary in terms of their sources of information.3 Human Rights Treaties and the IAEA, for example, rely to a large degree on experts; the Human Rights Treaty Bodies appoint Special Rapporteurs and the HRC hears evidence from CSOs and human rights organisations, while IAEA inspections are carried out by inspectors chosen from a pool of hundreds of international civil servants with technical expertise. With regard to CSO engagement, experts note that relying on civil society for this function depends on the existence of a vibrant CSO ecosystem in place that engages in the topic effectively. In terms of county visits, while some mechanisms grant powers to assigned entities to conduct these, some are conducted with country consent, such as in the Human Rights Treaties and the International Criminal Court, while the IAEA and Chemical Weapons Convention have the authority to conduct unconsented visits.3
Experts emphasised the importance of avoiding duplication in the global health regime and streamlining the monitoring of the pandemic agreement with other mechanisms. They noted the need to clarify how any mechanisms would interact with Joint External Evaluations (JEE), Universal Health and Preparedness Review (UHPR) and the Global Preparedness Monitoring Board (GPMB).
Proposal for an independent monitoring Committee
Based on the literature review and feedback from experts, this section puts forward principles to underpin the design of an independent monitoring committee (figure 1) for the implementation of the pandemic agreement.
The pandemic agreement should establish, as part of its institutional arrangements, an independent monitoring committee, tasked with producing regular assessments of state parties’ compliance with the pandemic agreement and the timeliness, completeness and accuracy of self-reporting. It would mainly be responsible for highlighting non-compliance to the body responsible for the enforcement of the legally binding elements of the agreement.
The independence of the monitoring body is important to allow it the freedom to call countries out on a lack of compliance or poor reporting. It would therefore need to be free from the political and financial pressures brought about by intergovernmental bodies and donors.
Soft incentives would be the most suitable enforcement mechanism, to avoid the harms of sanctions and benefits-based incentives. This could occur by using the gaps highlighted in the committee’s assessments to inform the allocation of the technical and material resources required for countries’ compliance. Reputational incentives would be provided by the committee reporting to the pandemic agreement COP as well as to a body consisting of or representing heads of state, such as WHO Member States (through the World Health Assembly) or the proposed Global Health Threats Council.26 The broad political leadership provided by such a body would also be consistent with the necessary whole-of-society and whole-of-government approach in PPR.
To provide a ‘check-and-balance’ function on the quality and accuracy of countries’ self-reports on their obligations, the committee would triangulate self-reporting with other public and private sources to identify gaps and inconsistencies. This could include shadow reports by CSOs and UN agencies, confidential reports from the public, including whistleblowing, consented country visits and direct confidential inquiries to state parties to the agreement.
The committee would share reports transparently into the public domain to promote public accountability, highlighting ‘best practice’, as a positive reputational incentive and to promote mutual learning, which would complement a ‘naming and shaming’ or reputational risk approach.
Sources of information
The committee would triangulate state self-reporting with other sources, including:
Reporting for the IHR.
State Party Self-Assessment Annual Reporting (mandatory).
JEE (a voluntary peer-review reporting mechanism, which is increasingly adopted).
The Universal Health and Preparedness Review (UHPR) (currently under pilot, proposed to be voluntary, likely to be comprehensive).
The Global Health Security Index (a civil society initiative).
Community-based/national CSOs focusing on PPR (shadow reporting; currently uncommon).
Although reports by the following types of organisations are not specific to PPR, their country reports may contain relevant details for the committee:
Intergovernmental organisations, such as the IMF, World Bank, UN agencies.
Other global organisations and foundations, such as the Gates Foundation, Gavi, the Global Fund.
Community-based/national CSOs that focus on health, societal or economic issues that may indirectly contain relevant information for PPR and the pandemic agreement.
The World Bank-hosted Pandemic Fund will likely require recipient countries to conduct reporting, which could also serve as input.
To maintain independence and promote a diversity of perspectives, the main criteria for committee membership would be to ensure a range of expertise, spanning public health, economics, social sciences, law, trade and finance. Membership would also aim for geographical and gender balance.
To maintain the political authority required for such a committee, members would be selected by the pandemic agreement COP.
Terms of reference
The committee would need to generate timely and relevant reports with regular follow-ups where necessary. It should, therefore, meet regularly (eg, quarterly) to review state parties’ compliance with and reporting on the pandemic agreement. The committee would determine the frequency at which states would undertake future reporting through a risk-rating system based on the assessments. Public accountability would be encouraged through accessible, digestible summaries, akin to, for example, the ALMA Scorecard or the GlobalChild Heatmap.27 28
Secretariat and hosting
A small, dedicated secretariat would support the committee by working on its behalf to collect, collate and prepare analyses and reports. It may also solicit additional information from states, at the request of the committee, through direct inquiries and arranging on-site visits.
The hosting arrangements for the committee and its secretariat should promote independent decision-making. This requires ring-fenced, unconditional, up-front and sustained funding by an international or multilateral body. Operations must be completely firewalled from any institution with a vested interest in supporting countries on PPR or that might be compromised in its activities if seen to be associated with a ‘negative’ report.
Focus of mandate and interaction with other structures
The committee would be limited to assessing countries’ compliance with and reporting on the legally binding elements of the pandemic agreement. It would not, therefore, conduct surveillance or outbreak investigations and, although it may inform technical support, would not provide it itself.
It would also seek to complement, rather than duplicate or undermine, other structures in the global health ecosystem. The committee would, therefore, mainly rely on existing data rather than requiring a lot of new data. For example, the IHR and UHPR are likely to continue as distinct entities that may inform the work of, but would be completely separate from, the pandemic agreement and associated bodies, such as this proposed committee. The GPMB is mainly concerned with preparedness at the global level, which is defined as more than the sum of national preparedness. The committee’s focus on countries’ performance would, therefore, not overlap with the GPMB’s mandate.
An accountability framework for the agreement can only be as ambitious as countries’ negotiated obligations. The effectiveness of an accountability framework will, therefore, be limited by the agreements that member states commit to through the INB. An effective accountability framework is necessarily dependent on member states agreeing to commitments that tackle those challenges. We also note the vast power injustice which demands a transformative shift in resources and power from the global north to the south and which would need to be accounted for in a design of a monitoring system for the pandemic agreement. Our analysis draws lessons from a range of international treaties and mechanisms, employing a broad range of literature and interviewing experts representing a breadth of perspectives and backgrounds.
There are a few limitations to this analysis. First, the proposal is limited, at this point, to inform the principles of an accountability framework for the pandemic agreement. A more detailed design would require a more detailed study. Second, having not recorded interviews, we were not able to take transcripts and, therefore, could not use direct quotes by experts. Results were recorded by two interviewers and codified in an Excel table, to mitigate a risk of recall bias in the findings and analysis. Third, to broaden the analysis and draw lessons, we analysed selected mechanisms outside of global health. The mechanisms were selected based on expert recommendations and thus, we may have missed others with relevant design elements for an accountability framework for the pandemic agreement.
There have been significant failures in PPR due, in part, to countries not fulfilling internationally agreed obligations. It is, therefore, important to consider how to avoid this shortcoming in future agreements, such as the proposed pandemic agreement. Our proposal, which uses lessons learnt from a broad range of international agreements and governance mechanisms, identifies key design principles for an independent monitoring mechanism for the pandemic agreement. We propose the establishment of an independent monitoring committee to assess countries’ compliance with and reporting on the pandemic agreement. It would be made up of independent experts, supported by a small, independent secretariat, have adequate ‘no strings attached’ financing, have access to a broad range of information sources, and be able to publish its findings regularly and without interference.
Data availability statement
All data relevant to the study are included in the article or uploaded as supplementary information.
Patient consent for publication
The authors would like to thank Kimberly Peek and Tiffany Nassiri-Ansari for their help with preparation of the manuscript.
Handling editor Seye Abimbola
Contributors LH, SL and NRS conceptualised the analysis. LH conducted the literature review. LH, SL and NRS conducted interviews. LH, SL, EH, CM and NRS contributed to the writing, provided advice and comments and reviewed the manuscript. LH, SL, EH, CM and NRS approved the final version for submission.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.