Article Text
Abstract
Objective To assess the validity of the South African Triage Scale (SATS) in four Médecins Sans Frontières (MSF)-supported emergency departments (ED, two trauma-only sites, one mixed site (both medical and trauma cases) and one paediatric-only site) in Afghanistan, Haiti and Sierra Leone.
Methods This was a retrospective cohort study conducted between June 2013 and June 2014. Validity was assessed by comparing patients’ SATS ratings with their final ED outcome (ie, hospital admission, death or discharge).
Results In the two trauma settings, the SATS demonstrated good validity: it accurately predicted an increase in the likelihood of mortality and hospitalisation across incremental acuity levels (p<0.001) and ED outcomes for ‘green’ and ‘red’ patients matched the predicted ED outcomes in 84%–99% of cases. In the mixed ED, the SATS was able to predict an incremental increase in hospitalisation (p<0.001) across both trauma and non-trauma cases. In the paediatric-only settings, SATS was able to predict an incremental increase in hospitalisation in the non-trauma cases only (p<0.001). However, 87% (non-trauma) and 94% (trauma) of ‘red’ patients in the mixed-medical setting were overtriaged and 76% (non-trauma) and 100% (trauma) of ‘green’ patients in the paediatric settings were undertriaged.
Conclusion The SATS is a valid tool for trauma-only settings in low-resource countries. Its use in mixed settings seems justified, but context-specific assessments would seem prudent. Finally, in paediatric settings with endemic malaria, adding haemoglobin level to the SATS discriminator list may help to improve the undertriage of patients with malaria.
- Health Systems evaluation
- Trauma
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Key questions
What is already known about this subject?
Emergency medical care (EMC) is recognised as being one of the core components of a horizontal approach to improving population health in low to middle-income countries (LMICs). Unfortunately, EMC is seldom seen as a priority in LMICs. One of the major challenges is the extreme mismatch between the demand for EMC and resource capacity. Triage may offer a simple and cost-effective mechanism for managing this problem.
There are few triage scales designed specifically for use in LMICs; the South African Triage Scale (SATS) is one of them. The SATS has been assessed extensively in South Africa, but its performance across a spectrum of different LMIC settings, mainly non-sub-Saharan African and trauma-only settings, has not been adequately evaluated.
What are the new findings?
This is the first study to look at the use of the SATS and its validity in Afghanistan, Sierra Leone and Haiti.
Recommendations for policy?
The findings of our study have several practice and policy implications. Given that the SATS was found to be highly valid in two different trauma settings in Haiti and Afghanistan, the implication is that this tool would be of value in other low-resource trauma settings too.
Introduction
Emergency medical care (EMC) is recognised as being one of the core components of a horizontal approach to improving population health in low to middle-income countries (LMICs).1 A substantial number of the conditions that contribute to the burden of disease in these countries could be alleviated through the provision of effective EMC. Unfortunately, however, EMC in LMIC settings is seldom considered a priority: EMC systems remain underdeveloped and under-resourced. One of the major challenges is around the extreme mismatch between the demand for EMC and resource capacity. Triage may offer a simple and cost-effective mechanism for managing this challenge.
Triage aims to determine a patient’s urgency for care (defined as their acuity level) in order to separate critically ill patients, who need immediate life-saving interventions, from patients who need medical attention but can safely wait to be seen.2 It is one of the core requirements for the provision of effective EMC and has been shown to reduce morbidity and mortality.3
There are few triage scales designed specifically for use in LMICs; the South African Triage Scale (SATS) is one of these: a four-level triage scale that colour codes patients as follows: (1) red—emergency; (2) orange—very urgent; (3) yellow—urgent; or (4) green—routine.4 The SATS has been assessed extensively in South Africa, but its performance across a spectrum of different LMIC settings, particularly non-sub-Saharan African and trauma-only settings, has not been adequately assessed.
To be of value, a triage tool must demonstrate good validity, that is, the acuity ratings assigned using the triage scale must closely reflect patients’ true acuities.5 Assessing the validity of a triage tool is inherently challenging due to the absence of a gold standard. Previous studies have either used surrogate markers such as hospital admission, mortality or resource utilisation as a proxy for true patient acuity6–10 or they have validated triage ratings assigned by emergency department (ED) staff for a series of simulated cases against ratings obtained from an expert panel.5 11 Both methods have their strengths and limitations. In particular, the use of some surrogate markers relies on reliable and accurate record keeping and standardisation of clinical care, which is often lacking in LMIC settings.
Médecins sans Frontières (MSF), an international medical humanitarian organisation, provides free medical care, including emergency care, to vulnerable populations in many LMICs. Since 2011, MSF-Operational Centre Brussels has been using the SATS in projects where it provides emergency care. Aside a project in Pakistan,11 the validity of the SATS has not been assessed in any of these settings.
Unlike many public EDs in LMICs, MSF-supported EDs are more suited for testing and comparing the validity of a triage tool using ED outcomes as a proxy for true acuity. This is because a standardised package of ED care is implemented; on-site support, training and supervision are ensured; and robust and standardised data collection systems are in place. As such, we set out to assess the validity of the SATS in four MSF emergency care settings (two trauma sites, one mixed (both trauma and medical cases) site and one paediatric-only site) in Afghanistan, Haiti and Sierra Leone, using ED outcomes (discharge, hospitalisation and death) as our reference standard.
Methods
Study design
This was a retrospective cohort study. Validity was assessed by comparing patients’ recorded SATS acuity ratings with their final ED outcome (hospital admission, death or discharge). The rationale for using ED outcomes as a reference standard for this validation was based on the logic that (1) moving incrementally from low to high acuity levels (‘green’ to ‘red’), the SATS would demonstrate an increasing trend in the likelihood of mortality and hospitalisation; (2) SATS triaged ‘routine’ (green) patients should not die or require hospitalisation; and (3) SATS triaged ‘emergency’ (red) patients would require hospitalisation or may die.
Ethics
Approval was obtained from the national ethics bodies in Afghanistan, Haiti and Sierra Leone, from the MSF Ethics Review Board and from the University of Cape Town.
Study setting and population
The study involved four MSF sites: two trauma centres in Kunduz, Afghanistan and Port-au-Prince, Haiti; a mixed ED in Martissant, Haiti; and a paediatric ED in Bo, Sierra Leone.
Kunduz Trauma Centre, Afghanistan
Kunduz province is located in North-Eastern Afghanistan, with a population of approximately one million. It has experienced a large amount of active conflict and sees high rates of trauma, mainly due to violence and road traffic accidents. The MSF Kunduz Trauma Centre was opened in August 2011 to fill a gap in the provision of trauma care in the area. At the time of the study, the centre had 92 inpatient beds and offered emergency, orthopaedic, surgical, physiotherapy and intensive care. The ED had a caseload of over 1800 patients with trauma per month. The SATS has been in use in the ED since the centre opened. In October 2015, the hospital was bombed and destroyed by US-led coalition forces killing patients and 14 MSF staff members, in the largest loss of life for MSF ever.
Tabarre surgical and trauma centre, Haiti
The MSF Tabarre trauma centre is located in an industrial area of Port au Prince—Haiti’s capital—and serves a catchment population of about one million. The city has a high burden of trauma emergencies related, in particular, to high numbers of road accidents and growing urban violence. The centre is a modular hospital that started providing specialised care for trauma and acute surgical conditions in February 2012. At the time of the study, the hospital had 107 beds and offered emergency, surgical, orthopaedic and intensive care, including inpatient and physiotherapy services. It saw an average of 793 patients with trauma per month. The SATS was implemented from the time the centre opened.
Martissant clinic, Haiti
Situated in the busy urban area of Martissant in Port au Prince, this MSF clinic serves a catchment population of 1.2 million people. The facility has an ED with a short-stay ward and when needed has the ability to convert a section into a cholera treatment centre. It deals with a mixed medical caseload and treats approximately 4900 patients per month. The SATS was introduced in 2012.
Gondama Referral Centre, Bo, Sierra Leone
At the time of the study, Gondama Referral Centre (GRC) was a mother and child MSF hospital, admitting children aged 15 years and below, situated close to the town of Bo in the Southern Province of Sierra Leone, approximately 250 km from the capital city of Freetown. The hospital included a paediatric-only ED, an inpatient department with wards dedicated to malnutrition and Lassa fever and a separate maternity unit. At the time of the study, the hospital saw approximately 500 patients per month and served a catchment population of 300 000. The paediatric version of the SATS was implemented at GRC’s ED during 2012.
SATS and its use in the EDs
The SATS is a four-tiered triage tool which has been extensively described elsewhere.12 The colour categories show the urgency into priority 1: red—‘emergency’ (to be seen immediately); priority 2: orange—‘very urgent’ (to be seen within 10 min); priority 3: yellow—‘urgent’ (to be seen within 60 min) and priority 4: green—‘routine’ (to be seen within 240 min). The SATS also allocates the colour blue to ‘dead on arrival cases’ (MSF has used the colour black to denote dead on arrival cases due to cultural norms in certain countries).
Patient triage was carried out by registered nurses at each site, and training in triage and use of the SATS was covered in a 1-day training session.
Study protocol
The study included all patients who presented at the study sites between June 2013 and June 2014. Patients were excluded if they were declared dead on arrival (and therefore not triaged). Patients were also excluded if they were referred for care, or absconded from care without medical permission, because of not being able to use these two outcomes as logical proxies for acuity level.
Data collection
Data pertaining to the study were sourced from the electronic ED registers at each site and extracted into an Excel database. Variables included age, sex, triage acuity (green, yellow, orange or red), ED outcome (discharged, hospitalised or died) and type of case (trauma, non-trauma).
Analysis
Reference standards (ie, predicted ED outcomes) according to acuity were as follows. (1) Moving incrementally from a low to high acuity (‘green’ to ‘red’): an increasing trend in the likelihood of mortality and hospitalisation. (2) ‘Routine’ (green) patients: discharge; no hospitalisations or deaths. (3) ‘Emergency’ (red) patients: hospitalisation or death; no discharges
Trends in mortality and hospitalisation over incremental acuity levels were assessed using the χ2 test for trend. Additionally, a univariate analysis was performed to determine the relative risk of hospitalisation by acuity level. A log-binomial model was used as the first approach for this analysis, but as the model failed to reach convergence, a Poisson regression with robust variance estimator model was used instead13 (the latter was not performed for mortality due to there being zero deaths for some acuity levels). For the last two reference standards, frequencies and proportions were calculated to assess the distribution of ED outcomes for ‘green’ and ‘red’ patients. Undertriage was calculated as the proportion of ‘green’ patients requiring hospitalisation or dying, and overtriage, the proportion of ‘red’ patients being discharged. Accepted thresholds for undertriage and overtriage were set at 10% and 50%, respectively, according to the American College of Surgeons Committee on Trauma.14 In Martissant and GRC, the reference standards were considered in relation to trauma and non-trauma ED presentations. The level of significance was set at p=0.05 throughout. Data were analysed using the STATA/IC V. 12.0 software (Stata, Texas, USA).
Results
During the study period, the total numbers of ED patients presenting at Kunduz, Tabarre, Martissant and GRC were 19474, 7706, 56 919 and 8190, respectively. Data on sex, outcome, acuity or type of case were missing for 3 patients at Kunduz and 15 patients at Martissant, and as such these patients were excluded from the analysis. Further patients were excluded if they were dead on arrival or if they were either referred for or absconded from care: 1845 (9%) at Kunduz, 286 (4%) at Tabarre, 2173 (4%) at Martissant and 301 (4%) at GRC.
Patient characteristics and their ED outcomes by acuity level
Table 1 shows the characteristics of the patients included in the analysis and table 2 shows their ED outcomes by acuity level.
Performance of the SATS according to the reference standards
In the trauma-only settings of Kunduz and Tabarre, the SATS performed well across all acuity levels: it accurately predicted an increase in the likelihood of mortality and hospitalisation moving from low to high acuity levels (p<0.001) (tables 2 and 3 and figure 1), and ED outcomes for ‘green’ and ‘red’ patients matched the predicted ED outcomes in 84%–99% of cases (table 4). Undertriage for ‘green’ patients and overtriage for ‘red’ patients did not exceed the acceptable thresholds of 10% and 50%, respectively.
In the mixed setting of Martissant, the SATS accurately predicted an increasing trend in mortality and hospitalisation moving from low to high acuity levels for both trauma and non-trauma cases, p<0.001 (tables 2 and 3 and figure 1. For ‘green’ patients, ED outcomes matched the predicted outcome in over 99% of cases (indicating virtually no undertriage), whereas predicted ED outcomes for ‘red’ patients were only matched in 12% of non-trauma cases and 6% of trauma cases (table 4); most ‘red’ cases were discharged (table 3) indicating a situation of overtriage that exceeded the acceptable threshold of 50%. The most common ED presentations among ‘red’ patients who were discharged (n=1024) were fever without identified cause (22%), accidental trauma (13%), lower respiratory tract infection (11%), upper respiratory tract infection (8%) and asthma (6%).
In the paediatric setting of GRC, the SATS was able to accurately predict a significant increase in the likelihood of mortality moving from low to high acuity levels for patients with trauma and non-trauma patients, p<0.001 (table 2). The trend in hospitalisation moving from low to high acuity levels however was marginal (for patients with trauma, there was no significant trend and for non-trauma patients although statistically significant we would question the clinical significance) and largely distorted by the high proportion of ‘green’ patients’ (the reference group) being hospitalised. The SATS performed well for ‘red’ patients with predicted ED outcomes being matched in 100% and 98% of patients with trauma and non-trauma patients, respectively (overtriage was therefore well below the acceptable threshold of 50%), but no ‘green’ patients with trauma and only 24% of ‘green’ non-trauma patients were discharged in accord with what was predicted (table 4); the majority of ‘green’ patients were hospitalised indicating a situation of undertriage that exceeded the acceptable threshold of 10% (table 2). Among ‘green’ patients who were hospitalised (n=547), more than half had confirmed severe malaria (33%) or confirmed uncomplicated malaria (25%).
Across the four sites, gender did not affect the predicted ED outcome (data not shown).
Discussion
This is one of few studies assessing and comparing the validity of the SATS in different low-resource settings—especially non sub-Saharan African settings—and the first to assess validity in trauma settings. The SATS demonstrated high validity in the trauma settings of Kunduz and Tabarre but in the mixed setting of Martissant and the paediatric setting of Gondama, it demonstrated a tendency to overtriage at the ‘red’ end of the scale in Martissant and to undertriage at the ‘green’ end of the scale in Gondama.
The main strengths of the study were that (1) it was multisite; (2) it included large numbers of patients; (3) it involved the analysis of real-life ED cases representative of the setting, rather than generic paper-based cases; and (4) the study period covered one whole year, thus taking account of seasonal variations in types of ED cases and disease burden.
The main study limitations were around the use of ED outcomes as a proxy for true acuity. First, while ED outcomes are accepted as being a suitable proxy for true acuity,6 8 10 given the absence of a gold standard, various factors may have confounded this relationship including inaccurate measurement and recording of patient acuity, inaccurate outcome reporting and suboptimal ED care resulting in a more severe ED outcome than would otherwise have been predicted. As it was, we believe that these factors were minimised as far as possible through the provision of standardised ED care, adequate resources (human and material), on-site support, supervision and regular training of ED staff, and robust data collection systems. Second, the reference standards (ie, the predicted ED outcomes) assigned for ‘green’ and ‘red’ patients may have failed to take account of anomalies in discharge and hospitalisation criteria related to certain patient conditions, leading to an overestimation of undertriage and overtriage. Third, because we were unable to predict specific ED outcomes for ‘yellow’ and ‘orange’ patients, undertriage and overtriage could not be estimated for these middle acuity levels. Finally, in the absence of any reference standards indicating how steep an increasing trend in mortality or hospitalisation across incremental acuity levels should be, we cannot use this to try to quantify the relative validity of the SATS in any way. A positive trend, however, provides evidence that the SATS is able to discriminate well between successive acuity levels and supports its validity.
To date, very few studies have assessed the validity of the SATS in low-resource settings; among those that have, only one has assessed it in a non-sub-Saharan African setting11 and none in any trauma-only settings. Furthermore, the study design used in most of these studies has generally involved comparing the triage ratings assigned by ED staff for a series of simulated paper-based cases with those obtained from an expert panel.6 15 This methodology has several limitations: (1) non-verbal patient cues and contextual information can introduce a potential interpretation bias, (2) the simulated cases used do not always represent the patient populations and disease burdens seen in the study setting and (3) it is difficult to decipher whether any identified discrepancies between ED staff ratings and the expert panel ratings are in fact a reflection of poor validity of the SATS or because ED staff are not applying the triage tool accurately.11
There is only one other study, conducted in South Africa, that has assessed the validity of the SATS using surrogate markers as a proxy for true acuity.6 In this study, hospital admission and discharge were used as the outcome markers for validating a revised version of the paediatric SATS. Validity indicators were determined based on the assumption that all discharges would comprise ‘green’ patients and that all hospital admissions would comprise ‘yellow’, ‘orange’ or ‘red’ patients. In our study, for ‘green’ and ‘red’ patients, we essentially applied this same assumption (ie, that among ‘green’ patients there would only be discharges and that among ‘red’ patients, there would be no discharges). However, for the acuity levels ofyellow and orange, we refrained from hypothesising that there would be no discharges; this, on account of the type of ED presentations that could potentially fall under these acuities. Instead, we assessed these middle acuities by applying the logic that moving incrementally from a low to high acuity, an increasing trend in the likelihood of mortality and hospitalisation would be observed. This same logic has been applied and demonstrated in other triage validity studies.7 9 16 17 We feel that this trend analysis, in combination with the specific reference standards assigned for ‘green’ and ‘red’ patients at either end of the scale, provides a more rational way of evaluating the validity of a triage scale when using ED outcomes.
In the trauma-only settings of Kunduz and Tabarre, the SATS demonstrated excellent construct validity. In the mixed medical setting of Martissant, however, while the ‘green’ to ‘orange’ part of the SATS seemed to perform well, the ‘red’ end of the scale appeared to overtriage patients, that is, to overestimate their urgency for care. Given that this was not the case in the trauma settings of Kunduz and Tabarre, we had wondered whether this might be a phenomenon specific to non-trauma medical patients. This hypothesis seems unfounded however as high rates of overtriage among ‘red’ patients in Martissant were observed for both patients with trauma and non-trauma patients. An alternative explanation for this overtriage might instead relate to the set up at Martissant. Martissant is a clinic with only a small six-bed short-stay ward on site and therefore there is a strong drive to discharge patients as soon as possible. Anecdotal evidence suggests that a significant proportion of the discharged ‘red’ patients presented with conditions (accidental trauma, fever, respiratory conditions) which were justifiably ‘emergency’ in nature at the time of triage but which staff at the clinic were able to quickly stabilise so that patients could be safely discharged. It is also possible that in some cases there was unpredicted recovery of serious looking initial symptoms. This requires further investigation, but suggests that the construct of the SATS itself is not to blame for what appears to be a situation of overtriage. Further support for the validity of the SATS in this setting comes from the clear increasing trend in mortality and hospitalisation observed moving incrementally across acuity levels.
In Gondama, the ‘green’ end of the scale appeared to have a strong tendency to undertriage patients, that is, to underestimate their urgency for care. The ramifications of undertriage are more dangerous than those of overtriage, whereby sick patients do not receive the necessary medical attention as urgently as needed. At GRC, this tendency to undertriage patients was more common among patients with trauma compared with non-trauma patients. Given that patients with trauma only made up 1% of GRC’s total presentations, one possible explanation for this difference may be related to clinicians erring on the side of caution more for trauma cases (on account of having less experience of managing them), leading to patients being admitted even if this is not always necessary. This hypothesis is indirectly supported by the fact that there were no deaths among any of these ‘green’ patients with trauma. Had deaths been observed among this group of patients, this would more strongly point towards the fact that the seriousness of the patient’s condition had gone unrecognised, that is, they had indeed been undertriaged. At any rate, further investigation into the management of patients with trauma at GRC would be warranted to substantiate our speculation around this issue.
The bulk of the patients at GRC were non-trauma patients and more than half of these patients who were triaged as ‘green’ and then hospitalised had uncomplicated or severe malaria diagnosed after triage. Most of these patients had concurrent severe anaemia of <7 g/dL, and this was the primary reason for them being hospitalised. As the paediatric version of the SATS used in GRC does not include a measure for haemoglobin, severe anaemia no doubt went ‘unnoticed’ at the time of triage, explaining why these patients were undertriaged. To mitigate this, we would recommend adding haemoglobin cut-off values to the SATS discriminator list and reassessing the performance of the SATS. If this modification is able to negate the undertriage of all ‘green’ hospitalised patients with malaria, undertriage among green patients would theoretically reduce from 88% to 32% which is a significant improvement. However, it still leaves the undertriage rate above the acceptable threshold of 10% and this needs careful investigation.
The findings of our study have a number of practice and policy implications. First, given that the SATS was found to be highly valid in two different trauma settings in Haiti and Afghanistan, the implication is that this tool could be of value in other low-resource trauma-only settings too. Trauma is a leading cause of global mortality (injuries account for approximately 5.8 million deaths annually) with 90% of these deaths occurring in LMICs.18 Minimising the morbidity and mortality related to injuries relies on timely and effective emergency care and the SATS could provide a simple and cost-effective mechanism by which to facilitate this.
Second, although there was some questionability about the performance of the SATS in the mixed setting of Martissant, we believe that this is largely related to the set up at the Martissant clinic itself, rather than the construct of the SATS. As such, implementation of the SATS in other low-resource mixed ED settings would seem justified, although context-specific assessments in such settings would still be prudent.
Third, in paediatric settings with endemic malaria, adding anaemia to the discriminator list could help ensure that patients with malaria are given greater priority.
Finally, when assessing the validity of a triage scale using ED outcomes as a proxy for true acuity, we would recommend using a trend analysis of mortality and hospitalisation across incremental acuity levels, in combination with specific reference standards for the extreme acuity levels of the scale.
Conclusion
The SATS is a valid triage tool for prioritisation of patients with trauma in low-resource settings. Its use in mixed ED settings seems justified, but context-specific assessments of its performance would nonetheless be prudent. In paediatric settings with endemic malaria, adding haemoglobin levels to the SATS discriminator may help to improve the undertriage of patients with malaria.
Acknowledgments
We would like to extend our condolences to the families of the staff and patients who lost their lives in Kunduz Hospital (one of our study sites). The hospital was bombed and destroyed by US coalition forces in October 2015. We would like to acknowledge and thank all the staff both nationally and internationally who helped with this study.
References
Footnotes
Contributors MD, PV, MT, LW and KT-S designed, analysed and interpreted the study and data.
YN, PJ and MS were the project leads in Sierra Leone. MN and AR were the project leads in Afghanistan. OG and PRJ were the project leads in Haiti.
All authors contributed to the revision of the final article.
Funding Mohammed Dalwai is supported by a grant from the Medical Research Council of South Africa, under the MRC Clinician Researcher Program
Competing interests No, there are no competing interests.
Patient consent Routine anonymised patient data was obtained from MSF sites.
Ethics approval Approval was obtained from the national ethics bodies in Afghanistan, Haiti and Sierra Leone, from the MSF Ethics Review Board and from the University of Cape Town.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement Data sharing on request