Article Text
PDF
Abstract
Antimicrobial resistance (AMR) is an urgent and growing global health concern, and a clear understanding of existing capacities to address AMR, particularly in low-income and middle-income countries (LMICs), is needed to inform national priorities, investment targets and development activities. Across LMICs, there are limited data regarding existing mechanisms to address AMR, including national AMR policies, current infection prevention and antimicrobial prescribing practices, antimicrobial use in animals, and microbiological testing capacity for AMR. Despite the development of numerous individual tools designed to inform policy formulation and implementation or surveillance interventions to address AMR, there is an unmet need for easy-to-use instruments that together provide a detailed overview of AMR policy, practice and capacity. This paper describes the development of a framework comprising five assessment tools which provide a detailed assessment of country capacity to address AMR within both the human and animal health sectors. The framework is flexible to meet the needs of implementers, as tools can be used separately to assess the capacity of individual institutions or as a whole to align priority-setting and capacity-building with AMR National Action Plans (NAPs) or national policies. Development of the tools was conducted by a multidisciplinary team across three phases: (1) review of existing tools; (2) adaptation of existing tools; and (3) piloting, refinement and finalisation. The framework may be best used by projects which aim to build capacity and foster cross-sectoral collaborations towards the surveillance of AMR, and by LMICs wishing to conduct their own assessments to better understand capacity and capabilities to inform future investments or the implementation of NAPs for AMR.
- Health systems evaluation
- Public Health
- Medical microbiology
- Infections, diseases, disorders, injuries
Data availability statement
All data relevant to the study are included in the article or uploaded as supplementary information.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Summary box
Despite a proliferation of tools designed to inform policy formulation and implementation or surveillance interventions to address antimicrobial resistance (AMR), there is an unmet need for easy-to-use instruments that provide a detailed overview of AMR policy, practice and capacity.
The Combating the threat of antimicrobial resistance in Pacific Island Countries (COMBAT-AMR) Assessment Framework represents a unique model of a systematic, cross-sectoral approach to assess AMR capacity across human and animal health microbiological laboratories, and hospital antimicrobial stewardship and water, sanitation and hygiene and infection prevention and control programmes within acute healthcare facilities.
The framework is flexible to meet the needs of implementers, as tools can be used separately to assess the capacity of individual institutions or as a whole to align priority setting and capacity building with National AMR Action Plans or national policies.
The COMBAT-AMR Assessment Framework has been used in multiple countries to support priority-setting, programme design, monitoring and implementation across human and animal sectors, and at the national level.
Use of the COMBAT-AMR Assessment Framework may be undertaken individually or with support and training from the COMBAT-AMR technical team to complete the tools and collaboratively develop priorities and further strategies to increase capacity.
Introduction
Antimicrobial resistance (AMR) represents one of the greatest challenges to public health globally, reducing the ability to prevent and treat infectious diseases.1 Infections due to resistant bacteria were associated with 4.95 million deaths and directly attributed to 1.27 million deaths in 2019.2 Projections on the impact of AMR outline the potential of up to 10 million deaths annually by 2050, and costs up to an estimated US$100 trillion, if significant action is not taken.3 However, the lack of high-quality data around the incidence of infections, prevalence of resistance and attributable mortality in low-income and middle-income countries (LMICs) introduces uncertainty into estimates of the burden of AMR.4 The overuse and misuse of antimicrobials including antibiotics, antivirals, antifungals and antiparasitics, in both humans and animals, have been the major factor to accelerate the emergence and spread of antimicrobial resistant organisms.1 This has been compounded in LMICs where under-resourced health systems, inadequate sanitation and infection prevention and control (IPC) systems within health facilities and communities, and a higher overall burden of infectious disease continue to drive increased reliance on antimicrobial use, and consequently, increased AMR.5 6 Contributing factors towards the spread of AMR globally occur across the human, animal and environmental health sectors and resultingly AMR requires a multidisciplinary and cross-sectoral One Health approach.7
Across Pacific Island Countries (PICs), there are limited data regarding existing mechanisms to address AMR, including national AMR policies, current infection prevention and antimicrobial prescribing practices, antimicrobial use in animals and microbiological testing capacity for AMR. The impact of the AMR in this region also remains poorly understood; however, there are some data to indicate that this is a major concern.8 9 A 2015 review of AMR surveillance identified varied capabilities among PICs across diagnostic testing capacity, infrastructure and workforce to implement AMR surveillance and monitoring, and inadequate or unenforced regulations governing accessibility and use of antimicrobials.10 Almost 10% of healthcare facilities in the western pacific region do not have a water supply, and almost two-thirds do not have basic supplies such as soap and water or alcohol-based hand rub to perform hand hygiene.11 These findings highlight the need to continue to develop capacity among PICs to support the prevention, diagnosis, surveillance and management of AMR.
Australia’s Department of Foreign Affairs and Trade’s Indo-Pacific Centre for Health Security funded the Combating the threat of antimicrobial resistance in Pacific Island Countries (COMBAT-AMR) programme to take a One Health approach to capacity-building and training activities to address AMR in four countries: Fiji, Papua New Guinea, Samoa and the Solomon Islands. COMBAT-AMR is implemented in partnership with government, National AMR Committees and key human and animal health stakeholders. COMBAT-AMR operates across five themes: IPC, antimicrobial stewardship (AMS), laboratory diagnosis and surveillance, animal health and One Health.
In planning COMBAT-AMR implementation, there was a need to assess current capacity and understand needs and priorities across the project themes. A number of tools and instruments exist to assess country and institutional capacity to detect and address AMR, and there have been previous evaluations of some of these tools against factors such as functionality, user experience and level of stakeholder engagement.12 13 However, there is currently no systematic, cross-sectoral capacity assessment model. Such an approach is particularly useful to build capacity and foster collaboration across multiple sectors and institutions, and monitor progress in implementation. Through COMBAT-AMR, we have developed a cross-sectoral AMR capability assessment structure (hereafter referred to as the ‘Framework’). The framework comprises five assessment tools, able to be implemented individually or in conjunction, to provide a detailed assessment of country capacity to address AMR. Four of the tools correspond to project themes of IPC, AMS, laboratory diagnosis and surveillance and animal health. The fifth is a national AMR situation assessment tool. The five tools are designed to be appropriate for countries at all levels of AMR capacity. The design of programme activities is responsive to local contexts and needs, and enables ongoing monitoring and benchmarking.
Here, we outline the process taken to develop and implement the framework and illustrate its application in the COMBAT-AMR target countries.
Development of the COMBAT-AMR capability assessment framework
Development of the five tools that comprise the framework was undertaken between September 2020 and April 2021, across three phases: (1) review of existing tools; (2) adaptation of existing tools and piloting; and (3) refinement and finalisation. The national AMR situation assessment tool was designed to take a One Health approach to reviewing national-level policies, practices and stakeholders related to AMR, disease surveillance and antimicrobial consumption and regulation.
The four tools that correspond with project themes are aimed at the organisational or facility level. The AMS tools and the IPC-water, sanitation and hygiene (WASH) tool are relevant to acute healthcare facilities. The human laboratory tool may be applied to all clinical microbiological laboratories; however, there are additional items specific to referral laboratories/sending laboratories. The animal health tool may be applied to any animal health microbiological laboratory.
Phase 1: review of existing tools
An initial literature review was undertaken to identify existing tools or instruments to assess AMR capacity across the project themes or at the national level. The literature review was undertaken with both peer-reviewed and grey literature; while the tools themselves were likely to be found in the grey literature, use of the tools may be reported in peer-reviewed literature (table 1). The COMBAT-AMR technical team was also asked to identify any known assessment tools or instruments in their relevant field.
Literature review search strategy
Tools and instruments were considered for inclusion if they:
had an explicit focus on assessment (eg, action plans, guidelines and recommendations, and implementation toolkits were excluded unless they had an assessment component) and
were aligned with the COMBAT-AMR themes (Animal health, AMS, IPC and WASH, Human health laboratory capacity and surveillance) or focused on national policy, practice and context relating to AMR.
The review identified the following existing assessment tools (table 2):
Identified tools to assess antimicrobial resistance (AMR) capacity at organisational and national levels
Supplemental material
Additional key guidelines and references which were not assessment tools but provided a more comprehensive overview of relevant items for inclusion are outlined in table 3:
Key guidelines and references which informed tool development
Assessment tools were primarily considered on the basis of:
Alignment with COMBAT-AMR themes of animal health, AMS, IPC and WASH, and human health laboratory capacity and surveillance or focused on assessment at the national level.
Scope: the focus of tools aligned with the project themes should be located at the organisational, rather than system or national level.
Complexity: simpler and easier-to-use tools were favoured over more complex ones.
Primarily quantitative: as the purpose of the tools for the project themes were for benchmarking and assessment purposes, items should be quantitative and scoreable. The national assessment tool was primarily qualitative, rather than quantitative in nature.
Following deliberation between COMBAT-AMR partners, the following tools were selected as the basis for the COMBAT-AMR assessment tools (table 4):
Rationale for existing tools selected to form the basis for COMBAT-AMR assessment tools
Phase 2: adaptation of existing tools
Phase 2 was an iterative process of refinement of the selected tools. For each of the themes, the relevant identified tools were reviewed for extensiveness and gaps. The identified instruments for the AMS and Human health laboratory surveillance tools were largely aligned with COMBAT-AMR project needs. In the case of the WHO Infection Prevention and Control Assessment Framework (IPCAF), the lack of items relating to WASH was seen to be a key gap. The IPCAF was adapted to include additional items from the WHO/UNICEF WASH FIT. National assessment for the animal health theme was included in the national assessment tool with a separate tool developed for assessment of animal health laboratories. The animal health laboratory capacity tool necessitated significant refinement in order to ensure that it was applicable to countries across differing levels of capacity. The Animal and Human health laboratory surveillance tools were adapted from the Fleming Fund AMR Surveillance Site and Laboratory Needs Assessment Tools for human and animal health, developed by Mott MacDonald with UKAid funding.
In order to allow for benchmarking and assessment of change over time, a scoring rubric was developed for each COMBAT-AMR theme assessment tool except the animal health laboratory capacity tool. Of the instruments selected to form the basis of the theme tools, only the IPCAF included an existing scoring rubric, and in this case the scoring needed to be adjusted to incorporate the additional WASH components. Key items with impact on potential AMR prevention or response capability and that were seen as representing best practice were selected to contribute to scoring, with non-scored items providing an overview of practice more broadly. For the national assessment tool, emphasis was placed on generating a snapshot of relevant policy and legislation, key stakeholders in AMR, regulation of antimicrobials and national priorities in AMR. For this reason, the national level tool was qualitative and did not use scoring.
The base tools each had different data capture methods. Some were Word or PDF-based, while others were Excel-based. In order to ensure consistency, each tool was entered onto the online application Research Electronic Data Capture (REDCap).14 REDCap allows for secure access and data collection, scoring and management and instruments can be accessed and used by individuals from multiple sites and institutions, making it appealing for projects where partners are geographically dispersed. Data can be entered into REDCap without an internet connection, then uploaded when a connection is available, facilitating data collection in areas of unreliable internet access. REDCap is used by nearly 6000 institutions and in 145 countries, representing strong potential for wider dissemination and uptake of the instruments. The scoring rubric for each tool was also incorporated into the REDCap versions. All tools except the AMS assessment tool incorporated significant branching to allow respondents to skip items that were not relevant to them while maintaining the integrity of the scoring system. This improved usability, reduced respondent burden and made the tools more responsive to contexts with a wide variety of capacities.
Guidance documents and instructions were developed for each tool. A data governance framework was established to support transparency and accountability in data ownership and usage.
Phase 3: piloting, refinement and finalisation
Piloting was undertaken in Australian institutions to further refine the overall process, provide estimates of the time necessary to complete the tools, improve clarity and flow of the tools and individual items, and ensure the appropriateness of scoring categories and suggested personnel to be involved in the assessment process. While the tools are developed primarily for use in LMIC, it was decided to pilot in Australian institutions as these facilities would likely be more developed and extensive, ensuring that the full range of capacities would be able to be captured. Pilots were undertaken in human public health and animal laboratories and in acute care hospitals (ie, tools were not piloted in aged care facilities, or community-based or general practices). After each pilot, feedback was collated and discussion was held between the individual(s) completing the tool and the technical team to clarify any feedback or queries, and to refine the tool. Piloting was concluded once no substantiative feedback or suggestions were received. One to three rounds of piloting were undertaken for each theme assessment tool. Detailed outlines of the feedback received and modifications made to the tools throughout the piloting process are found in online supplemental annex 1.
The technical team completed extensive review and optimisation on the national assessment tool; however, this tool was not piloted. Once piloting had been completed, the theme assessment tools underwent a last round of refinement and review to ensure all comments and feedback were addressed. The tools and all accompanying documents were then finalised. Following implementation into the COMBAT-AMR countries, the tools underwent an additional round of revision incorporating feedback from their use in LMICs. Feedback was given on logistical issues in completing the tools, comments on the size of the tools and time needed to complete and comments of question wording. These were incorporated into updated final versions of the tools.
Format and structure of the COMBAT-AMR situation assessment tools
An overview of the COMBAT-AMR assessment tools is provided in table 5. Each of the assessment tools comprises between one and three instruments. The tools can be used separately to assess particular aspects of an institution’s AMR capacity, or as a whole to provide a detailed, cross-sectoral overview of a country’s current AMR capacities, needs and priorities. Use of the National AMR Situation Assessment Tool in conjunction with the theme assessment tools allows countries to align investment and capacity building with National AMR Action Plans or national and regional strategies, and provides the basis for a cohesive programme of work that addresses AMR using a cross-sectoral approach. Completion of the tools should be collaborative and involve a range of stakeholders and involved parties, rather than being a single individual’s assessment.
Summary of the COMBAT-AMR assessment tools
The National AMR Situation Assessment Tool includes three data collection instruments: The National Antimicrobial Resistance Context Review, the National Antimicrobial Resistance Policy and Practice Assessment and the key informant interview schedule. These data collection tools have been designed to be flexible and minimise respondent burden, allowing for use with a variety of respondents and key informants and across countries. Topics covered by the national-level tools align with the COMBAT-AMR themes; the Strategic Objectives of the WHO Global Action Plan on AMR, the Tripartite AMR Country Self-assessment Survey; and the FAO Progressive Management Pathway for AMR.
The two components of the AMS assessment tool, the Antimicrobial Stewardship Gap Analysis Tool and AMS Facility Observation Checklist, work together to provide a robust assessment of a healthcare facility’s AMS processes, policies and practices. The AMS Gap Analysis Tool provides a detailed overview of practices to support investment and planning, and to capture progress towards best practice over time. The AMS Faculty Observation Checklist provides observable confirmation of facility AMS activities and practices.
The other three tools are each comprising a single instrument. The animal health laboratory capacity assessment tool and human health laboratory capacity assessment tool have a number of elements in common, each examining capacity in antimicrobial susceptibility testing and elements of laboratory function including reporting, sample referral pathways and operations, quality assurance and infrastructure. The animal health laboratory capacity assessment tool incorporates queries regarding species that clinical and healthy samples are received from, as well as questions regarding food and environmental samples. Items in the human health laboratory capacity assessment tool regarding pathogens versus antibiotic resistance combination align with the GLASS guide.15
The IPC WASH assessment tool covers elements such as availability and use of IPC and WASH guidelines, WASH system and infection surveillance and practices regarding staffing and bed occupancy. WASH items are included in most sections of the tool and examine the built environment and all relevant WASH components for healthcare facilities: water, sanitation, hygiene for all healthcare users, healthcare waste management and environmental cleaning.
Implementation and utilisation of the assessment framework in programme design and evaluation
Each assessment tool has been implemented in either one or two of the target COMBAT-AMR countries. Country 1 implemented all five assessment tools while country 2 implemented the national assessment tool and the animal health laboratory capacity assessment tool. Country 3 completed the AMS tools and partially completed the national assessment tool. Country 4 completed the IPC tool. Implementation of the theme assessment tools was undertaken within the target countries at the facility level. This was performed by key individuals at each of the different institutions whose professional roles corresponded to the project themes. Tools were completed online with significant support from the COMBAT-AMR country coordinator and the technical team.
Feedback regarding the tools from those institutions that completed the theme assessments was highly positive, and included comments that the scores for the different sections of the assessments corresponded with their own perceptions of their organisational capacity. However, feedback was also received that the human health laboratory capacity assessment tool was particularly burdensome to complete and required significant oversight and review from the COMBAT-AMR technical team.
Following tool completion, the information from the assessment is converted to quantitative scores to identify areas of strength and opportunities. This scoring is then discussed with relevant stakeholders to produce initial recommendations and identify key priorities for intervention or investment. Standardised summary reports are generated by COMBAT-AMR technical staff which includes an analysis of the strengths and opportunities for each participating institution. Following collaborative discussions of report findings, recommendations may be revised and workplans consisting of targeted activities are developed.
Reports were provided to and discussed with personnel within each institution who had completed data collection, and senior managers or government stakeholders. At this point, outstanding queries were able to be resolved and alignment or discrepancies in the institution’s scores and staff perceptions were discussed.
The summary reports guided identification of key areas of opportunities, provided a foundation for prioritisation of key areas for support, and contributed to development of project workplans. The scoring system provided a transparent means of identifying potential priorities and areas that represented particularly important targets for capacity building while the reporting process supported collaboration and stakeholder engagement in identification of priorities and project workplan development. Further gaps were identified through completion of the national assessment and subsequent stakeholder interviews to refine the COMBAT-AMR programme and workplans. While project implementation is still ongoing, the theme assessment tools will be repeated at the end of the project to provide an objective measure of project progress against identified priorities and outcomes.
Limitations
The COMBAT-AMR Assessment Framework provides a systematic approach to assessing AMR capacity, but can also be time consuming and necessitate a high level of technical skills or resources. Countries may therefore struggle to complete the tools independently and need additional support.
Initial piloting of the tools occurred only in Australia, within highly developed facilities. This was to ensure that the tools would be able to capture the full range of complexity across diverse contexts within LMICs. Further revisions were made to the tools incorporating comments from end-users following implementation in each of the four target countries.
The COMBAT-AMR Assessment Framework can be undertaken with support from the COMBAT-AMR technical team. Given that the theme tools are focused on an organisational level, they do not capture variation in capacity across institutions, nor do they take into consideration differences between wards or units within a hospital. However, the tools can be used at multiple institutions to provide a more nuanced understanding of AMR capacity at the whole of country level. As with all cross-sectional instruments, the tools provide a single snapshot in time, although multiple rounds of completion can be used to assess progress periodically. The tools may be applied broadly however certain fields will only apply to specific health facilities with an acute care component and microbiological laboratories. The scope of the tools is on pathogenic bacterial AMR with a focus on drug-resistance in healthcare settings aligning with the key high priority areas for global health funding of governance, disease surveillance and laboratory capacity. This underpinned the selection of the four programme themes. However, other areas relevant to AMR such as antimicrobial-consumption surveillance, education and awareness, and immunisation are not directly addressed through the tools and may be addressed through future work and the development of additional assessment tools. The animal health tool focuses primarily on animal health laboratories. This focus is due to recognition of existing limitations in AMR capacity in the animal health sector across LMICs and the resulting need to build laboratory capacity to improve the availability and quality of AMR surveillance data in the animal health sector. While environmental health is included in One Health, in LMICs the sector was viewed as out of scope for the development of the tools. Prior literature reviews and the authors’ own experience highlight that while One Health is a growing concept, the environmental sector is extremely limited in terms of infrastructure and resources. In the majority of LMICs, the environmental sector needs extensive investment in basic capacity building before focused planning in AMR is feasible.
Conclusions
AMR is an urgent and growing global health concern, and a clear understanding of existing capacities to address AMR, particularly in LMICs, is needed to inform national priorities, investment targets and development activities. Despite a proliferation of tools designed to inform policy formulation and implementation or surveillance interventions to address AMR, there is an unmet need for easy-to-use instruments that provide a detailed overview of AMR policy, practice and capacity.
The COMBAT-AMR Assessment Framework represents a unique model of a systematic, cross-sectoral approach to assess AMR capacity. The COMBAT-AMR Assessment Framework is available for use outside of the COMBAT-AMR project, and tools have been used in additional countries to support other implementation projects. The Framework is flexible to meet the needs of implementers, as tools can be used separately to assess the capacity of individual institutions or as a whole to align priority-setting and capacity-building with National AMR Action Plans or national policies. We therefore provide access to this suite of instruments to assess AMR capacity across a range of contexts and invite interest from policy-makers, practitioners and implementers that are in the process of assessing and building their capacity to manage AMR. Use of the COMBAT-AMR Assessment Framework may be undertaken individually or with support from the COMBAT-AMR technical team to complete the tools and collaboratively develop priorities and further strategies to increase capacity. Further information on how to access the COMBAT-AMR Assessment Framework may be found here: https://www.combatamr.org.au/project-activities/situation-and-needs-assessment-tool-development
Data availability statement
All data relevant to the study are included in the article or uploaded as supplementary information.
Ethics statements
Patient consent for publication
Ethics approval
Not applicable.
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Handling editor Seema Biswas
Twitter @asferdinand
Contributors ASF led the development of the tools and conceptualised the article. ASF and CM wrote the manuscript. ASF, CL, JM, GB, JG, MC, TP, SL, NT, AJ, AS, DC, AM, BPH and KB contributed to the design and piloting of the tools. ASF, CL, KB, KK, GT and BP led the implementation of the tools in the pacific. BPH and KB led the COMBAT-AMR project. All authors provided feedback to help shape the manuscript.
Funding This study was funded by Department of Foreign Affairs and Trade, Australian Government.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.