Discussion

This is one of few studies assessing and comparing the validity of the SATS in different low-resource settings—especially non sub-Saharan African settings—and the first to assess validity in trauma settings. The SATS demonstrated high validity in the trauma settings of Kunduz and Tabarre but in the mixed setting of Martissant and the paediatric setting of Gondama, it demonstrated a tendency to overtriage at the ‘red’ end of the scale in Martissant and to undertriage at the ‘green’ end of the scale in Gondama.

The main strengths of the study were that (1) it was multisite; (2) it included large numbers of patients; (3) it involved the analysis of real-life ED cases representative of the setting, rather than generic paper-based cases; and (4) the study period covered one whole year, thus taking account of seasonal variations in types of ED cases and disease burden.

The main study limitations were around the use of ED outcomes as a proxy for true acuity. First, while ED outcomes are accepted as being a suitable proxy for true acuity,6 8 10 given the absence of a gold standard, various factors may have confounded this relationship including inaccurate measurement and recording of patient acuity, inaccurate outcome reporting and suboptimal ED care resulting in a more severe ED outcome than would otherwise have been predicted. As it was, we believe that these factors were minimised as far as possible through the provision of standardised ED care, adequate resources (human and material), on-site support, supervision and regular training of ED staff, and robust data collection systems. Second, the reference standards (ie, the predicted ED outcomes) assigned for ‘green’ and ‘red’ patients may have failed to take account of anomalies in discharge and hospitalisation criteria related to certain patient conditions, leading to an overestimation of undertriage and overtriage. Third, because we were unable to predict specific ED outcomes for ‘yellow’ and ‘orange’ patients, undertriage and overtriage could not be estimated for these middle acuity levels. Finally, in the absence of any reference standards indicating how steep an increasing trend in mortality or hospitalisation across incremental acuity levels should be, we cannot use this to try to quantify the relative validity of the SATS in any way. A positive trend, however, provides evidence that the SATS is able to discriminate well between successive acuity levels and supports its validity.

To date, very few studies have assessed the validity of the SATS in low-resource settings; among those that have, only one has assessed it in a non-sub-Saharan African setting11 and none in any trauma-only settings. Furthermore, the study design used in most of these studies has generally involved comparing the triage ratings assigned by ED staff for a series of simulated paper-based cases with those obtained from an expert panel.6 15 This methodology has several limitations: (1) non-verbal patient cues and contextual information can introduce a potential interpretation bias, (2) the simulated cases used do not always represent the patient populations and disease burdens seen in the study setting and (3) it is difficult to decipher whether any identified discrepancies between ED staff ratings and the expert panel ratings are in fact a reflection of poor validity of the SATS or because ED staff are not applying the triage tool accurately.11

There is only one other study, conducted in South Africa, that has assessed the validity of the SATS using surrogate markers as a proxy for true acuity.6 In this study, hospital admission and discharge were used as the outcome markers for validating a revised version of the paediatric SATS. Validity indicators were determined based on the assumption that all discharges would comprise ‘green’ patients and that all hospital admissions would comprise ‘yellow’, ‘orange’ or ‘red’ patients. In our study, for ‘green’ and ‘red’ patients, we essentially applied this same assumption (ie, that among ‘green’ patients there would only be discharges and that among ‘red’ patients, there would be no discharges). However, for the  acuity levels ofyellow and orange, we refrained from hypothesising that there would be no discharges; this, on account of the type of ED presentations that could potentially fall under these acuities. Instead, we assessed these middle acuities by applying the logic that moving incrementally from a low to high acuity, an increasing trend in the likelihood of mortality and hospitalisation would be observed. This same logic has been applied and demonstrated in other triage validity studies.7 9 16 17 We feel that this trend analysis, in combination with the specific reference standards assigned for ‘green’ and ‘red’ patients at either end of the scale, provides a more rational way of evaluating the validity of a triage scale when using ED outcomes.

In the trauma-only settings of Kunduz and Tabarre, the SATS demonstrated excellent construct validity. In the mixed medical setting of Martissant, however, while the ‘green’ to ‘orange’ part of the SATS seemed to perform well, the ‘red’ end of the scale appeared to overtriage patients, that is, to overestimate their urgency for care. Given that this was not the case in the trauma settings of Kunduz and Tabarre, we had wondered whether this might be a phenomenon specific to non-trauma medical patients. This hypothesis seems unfounded however as high rates of overtriage among ‘red’ patients in Martissant were observed for both patients with trauma and non-trauma patients. An alternative explanation for this overtriage might instead relate to the set up at Martissant. Martissant is a clinic with only a small six-bed short-stay ward on site and therefore there is a strong drive to discharge patients as soon as possible. Anecdotal evidence suggests that a significant proportion of the discharged ‘red’ patients presented with conditions (accidental trauma, fever, respiratory conditions) which were justifiably ‘emergency’ in nature at the time of triage but which staff at the clinic were able to quickly stabilise so that patients could be safely discharged. It is also possible that in some cases there was unpredicted recovery of serious looking initial symptoms. This requires further investigation, but suggests that the construct of the SATS itself is not to blame for what appears to be a situation of overtriage. Further support for the validity of the SATS in this setting comes from the clear increasing trend in mortality and hospitalisation observed moving incrementally across acuity levels.

In Gondama, the ‘green’ end of the scale appeared to have a strong tendency to undertriage patients, that is, to underestimate their urgency for care. The ramifications of undertriage are more dangerous than those of overtriage, whereby sick patients do not receive the necessary medical attention as urgently as needed. At GRC, this tendency to undertriage patients was more common among patients with trauma compared with non-trauma patients. Given that patients with trauma only made up 1% of GRC’s total presentations, one possible explanation for this difference may be related to clinicians erring on the side of caution more for trauma cases (on account of having less experience of managing them), leading to patients being admitted even if this is not always necessary. This hypothesis is indirectly supported by the fact that there were no deaths among any of these ‘green’ patients with trauma. Had deaths been observed among this group of patients, this would more strongly point towards the fact that the seriousness of the patient’s condition had gone unrecognised, that is, they had indeed been undertriaged. At any rate, further investigation into the management of patients with trauma at GRC would be warranted to substantiate our speculation around this issue.

The bulk of the patients at GRC were non-trauma patients and more than half of these patients who were triaged as ‘green’ and then hospitalised had uncomplicated or severe malaria diagnosed after triage. Most of these patients had concurrent severe anaemia of <7 g/dL, and this was the primary reason for them being hospitalised. As the paediatric version of the SATS used in GRC does not include a measure for haemoglobin, severe anaemia no doubt went ‘unnoticed’ at the time of triage, explaining why these patients were undertriaged. To mitigate this, we would recommend adding haemoglobin cut-off values to the SATS discriminator list and reassessing the performance of the SATS. If this modification is able to negate the undertriage of all ‘green’ hospitalised patients with malaria, undertriage among green patients would theoretically reduce from 88% to 32% which is a significant improvement. However, it still leaves the undertriage rate above the acceptable threshold of 10% and this needs careful investigation.

The findings of our study have a number of practice and policy implications. First, given that the SATS was found to be highly valid in two different trauma settings in Haiti and Afghanistan, the implication is that this tool could be of value in other low-resource trauma-only settings too. Trauma is a leading cause of global mortality (injuries account for approximately 5.8 million deaths annually) with 90% of these deaths occurring in LMICs.18 Minimising the morbidity and mortality related to injuries relies on timely and effective emergency care and the SATS could provide a simple and cost-effective mechanism by which to facilitate this.

Second, although there was some questionability about the performance of the SATS in the mixed setting of Martissant, we believe that this is largely related to the set up at the Martissant clinic itself, rather than the construct of the SATS. As such, implementation of the SATS in other low-resource mixed ED settings would seem justified, although context-specific assessments in such settings would still be prudent.

Third, in paediatric settings with endemic malaria, adding anaemia to the discriminator list could help ensure that patients with malaria are given greater priority.

Finally, when assessing the validity of a triage scale using ED outcomes as a proxy for true acuity, we would recommend using a trend analysis of mortality and hospitalisation across incremental acuity levels, in combination with specific reference standards for the extreme acuity levels of the scale.