Phase 1: literature review for quality indicators

We searched peer-reviewed publications and the grey literature to determine commonly cited quality indicators for emergency care. We searched PubMed, MEDLINE, EMBASE and the Cochrane Library using the search term ((quality(Title) AND emergency(Title/Abstract)). Fifty-three articles were selected which included publications that both addressed emergency care quality indicators and enumerated a list of indicators. We also reviewed citation lists of included papers. We then further searched indicators for different specialty domains relating to emergency care, those published by professional societies in the UK, USA, Canada and Australia, and those from the Columbia University sidHARTe Programme11–14 16 (Systems Improvement at District Hospitals and Regional Training of Emergency Caredx - diagnosis (sidHARTe) Indicators for Acute Care in Low and Middle-Income Countries Toolkit, unpublished). All indicators listed were extracted from the articles and compiled.

After literature review, 2307 indicators were returned. The initial list of 2307 indicators contained many indicator duplicates, as well as indicators that had slightly different wording but were synonymous. Duplicates were removed, synonymous indicators were represented by a single indicator and final list of indicators (137) were categorised by the authors (MCB and EJCH) into Donabedian’s categories of structure, process and outcome.25 The categorised indicators were mapped to specific components of the patient encounter as defined by the WHO Emergency Care System Framework in order to break the indicators into more manageable blocks for discussion, and provide context and organisation for consensus conference participants.26

Phase 2: selection of clinical conditions and initial indicators

A diverse group of 32 physicians, clinical officers, nurses and administrators from 21 countries participated in the 2016 AFEM Consensus Conference. Participants were provided with a description of the working group goals and objectives, a briefing document describing the core tenets of clinical quality indicators and a template of the indicator matrix prior to the initial encounter day. Patient safety metrics in emergency care are being addressed in a separate process by WHO, and thus were not specifically addressed during this consensus session.

Priority clinical conditions associated with emergent presentations were identified and informed by the Global Burden of Disease project.27 Explicit principles were established for the selection of clinical conditions: the condition must occur with significant frequency, be associated with high morbidity and mortality, fall within the scope of emergency care, and with timely intervention lead to improved clinical outcomes.

Indicators associated with these conditions were then reviewed by the group according to the inclusion criteria described in table 1. Borderline indicators were marked and noted for clarification in further rounds.

Through consensus, the working group identified seven clinical conditions to prioritise in their indicator development (table 2). Overall, the identified conditions addressed nearly 75% of the mortality in Africa.28 Of note, we deliberately chose not to include prehospital services: they will be the subject of a separate AFEM indicator development process.

Indicators compiled from the literature review were discussed and modified based on the targeted conditions and pre-established inclusion criteria (table 1). We did not generate any de novo indicators in phase 1. The group started with a list of 137 indicators: 101 (74%) reached agreement and were included in the next phase. The working group identified that timeliness indicators must be specific to only the EU encounter. The group deemed quantitative ‘time to’ indicators could not be feasibly collected with current charting methods in Africa.

It was decided that while critical incident rates are important for quality improvement, they are difficult to define and measure within a minimum set of clinical quality indicators. Additionally, patient perception of care was also identified as an essential quality metric, but standardisation of patient satisfaction scores across various cultures and countries within Africa was beyond the scope of the recommendations of this study.

Phase 3: formal survey—expert clinician selection of indicators

Conditions and quality indicators were sent to a convenience sample of 38 experts from 17 countries, with extensive experience in the provision of emergency care across Africa: as in the prior round, participants were given instructions to follow criteria established in table 1 and to reflect the conditions selected in table 2. All participants completed the survey. Respondents ranked each indicator according to a 5-point Likert Scale (ranging from ‘strongly disagree’ to ‘strongly agree’). During this phase, respondents were given the opportunity to suggest additional de novo indicators they felt appropriate. Quality indicators with greater than 70% agreement of ‘agree’ or ‘strongly agree’ were selected to proceed to the next round.

Of the 101 indicators, 89 (88%) were selected for inclusion in the next phase.

Phase 4: expert review and ranking

A subsequent expert panel sampled from senior delegates at the 2016 African Conference on Emergency Medicine was convened to review and provide further refinement of the quality indicators selected by the formal survey. This diverse panel was composed of 22 clinicians practising emergency care in Africa. The panel first reviewed the indicator list, and removed those that did not meet group consensus. The panel was then asked to rank the different indicators in three domains: correspondence with level of emergency system development, feasibility of data collection and value to patients (1—lowest priority, 3—highest priority). ‘Correspondence with level of emergency system development’ was described as whether the indicator was valuable as a measure of the overall development of the emergency care system; for example, mortality in the first 24 hours from trauma might be a good indicator of how well the system is functioning whereas documentation of disposition is particular to the facility and would likely be a less reliable indicator of system development. A composite score for each indicator was generated, and they were ranked according to their importance as defined by the three domains.

Seventy-six of the 89 indicators achieved consensus; due to time constraints, indicators deemed most important for measuring facility processes (55) were subsequently rated in three domains and a composite score used for generation of a prioritised master list (online supplementary appendix 2).

Phase 5: indicator definition and refinement

Each indicator was assigned a standard profile that included an operational definition, numerator and denominator, and reference data sources for data capture. Standard definitions for each term are included in the meta-data, which will be provided to users of the indicators. Further required detailed clarification was accomplished via expert review. The profiles for selected outcome clinical quality indicators are described in table 3. Process, time-based and structure indicators are included as online supplementary tables A–D. Summary of selected clinical conditions and their associated outcomes are available in table 4.

Limitations

There are a number of limitations that deserve mention. There is a widespread lack of published indicators for emergency care in LMICs, especially in Africa. This may be due to the fact that emergency medicine as a specialty either does not exist or has been recently developed in most countries. In addition, in many LMICs robust emergency care systems are in their infancy, so they may not yet have agreed on national indicators for quality at facilities.

Another limitation relates to the search strategy used to identify indicators. While we attempted to be as inclusive as possible, a formal systematic review was not employed for the purposes of this consensus process. Thus, we may have missed some possible indicators for inclusion. In addition, the authors grouped indicators by structure, process, and outcome and mapped them to the WHO emergency care system (ECS) Framework to facilitate discussion and provide a contextual reference point. These indicators were then subsequently disassociated from the framework and Donabedian categories in the final ranking. Grouping them in such a manner could have influenced the way that participants selected and subsequently ranked indicators.

The clinical conditions prioritised in this process may not be representative of the most important clinical conditions affecting a specific region or area. The local burden of emergent conditions is still unknown in much of Africa;3 thus, Global Burden of Disease data were used as a proxy for the burden of emergency conditions to guide the process.25 Further work is necessary to define the essential emergency conditions requiring evaluation with quality metrics for a given region.

Lastly, while the authors attempted to be as inclusive of those in emergency clinical practice in Africa as possible, convenience sampling of participants at the 2016 AFEM Consensus Conference and then the subsequently engaged expert panel could have skewed results so as not to be representative of the larger experience of emergency care on the African continent. Participants in this phase represented 21 countries but, as with any consensus process, not all views may have been included. Group discussion was elicited and facilitated but this may have led to some participants conforming with the majority view and may not represent the full spectrum of opinions.35