Manchester Triage in Sweden – Interrater reliability and accuracy
Introduction
Triage, i.e., prioritizing and sorting at emergency departments, has been attracting increasing attention. In particular, older patients with multiple diagnoses are becoming more common (Goodacre et al., 1999, Beveridge et al., 1999, Palmquist and Lindell, 2000, Göransson, 2006). The prioritization carried out has, therefore, also become more complex. More parameters have been introduced for the order in which patients are treated (Baldursdottir and Jonsdottir, 2002, Fernandes et al., 2005, Göransson, 2006).
The triage models, which were developed during the 1990s have been refined and have become national standards in some countries (Fernandes et al., 2005, Worster et al., 2004). Australia was the first to introduce a triage model, “The National Triage Scale”. The model was developed by The Australasian College for Emergency Medicine in 1993. At the beginning of the 21st century its name was changed to “The Australasian Triage Scale” (ATS). In Canada a triage model was developed in the mid-1990s, which was based on the Australian model ATS, “The Canadian Emergency Department Triage and Acuity Scale” (CTAS). In the USA another triage model, “The Emergency Severity Index” (ESI) has been in existence since the end of the 1990s (Beveridge et al., 1998, Gilboy et al., 2005, McCallum Pardey, 2006).
This study focuses on a fifth national standard model, “Manchester Triage” (MTS) (Mackway-Jones, 1997) which is accepted as a standard at emergency departments in Great Britain, Holland and Portugal (Lipley, 2005). In this article we examine the interrater reliability and accuracy using MTS at emergency departments in Western Sweden where the model has been implemented. There have been no earlier studies of the method applicable to Western Sweden. There was, therefore, a need to research the case.
Section snippets
Background
At the beginning of the 21st century discussions began at emergency departments in Western Sweden with regards to implementing a common triage model. At this time most departments used the Swedish National Board of Health and Welfare criteria document which only provided three prioritization levels (Socialstyrelsen, 1994). When the waiting times increased due to an increased patient influx at the emergency departments in Western Sweden, discussions arose that led to the conclusion that the
Method
The research design was a prospective and descriptive survey based on simulated patient cases. The data was collected by asking triage nurses (n = 79) to assess simulated patient cases and determine a triage category according to the same principles used in their daily triage work. The cases had been given reference values by two expert panels. An analysis subsequently took place of the κ values, the accuracy, the over-triage and the under-triage.
Nine emergency departments in Western Sweden use
Results
Seven emergency departments took part in the study. The nurses (n = 79) assessed 13 patient cases each. A total of 1027 triage assessments were analyzed. There were 82% female and 18% male participants, and 91% were older than 25 years of age. The majority of the nurses had worked more than two years after completing their basic training.
The participating nurses from the seven emergency departments together presented an unweighted κ value of 0.61 (SD 95%, CI 0.57–0.65), a linear weighted κ value
Deviation from predicted category
The deviation from the predicted triage categories red and orange was nearly one out of 10. These results would suggest that patients in need of urgent care were identified as such at the emergency departments covered in the study. However, there is still a group of severely ill patients that are not assigned to the predicted categories.
The high percentages of over-triage and under-triage in the yellow and green categories implied that the order of priority was disrupted. It can be significant
Conclusions
In this MTS study we found good interrater reliability at emergency departments in Western Sweden. The accuracy was high (73%). The triage categories red and orange showed the highest values (92% and 91%), which imply that MTS could identify patients in need of early intervention in more than nine out of 10 cases.
This study indicates that the implementation of a structured clinical decision support system could increase the interrater reliability and the accuracy while decreasing over-triage. A
References (30)
- et al.
Reliability of the Canadian emergency department triage and acuity scale: interrater agreement
Annals of Emergency Medicine
(1999) - et al.
Triage nurses’ decisions using the national triage scale for Australian emergency departments
Accident and Emergency Nursing
(2000) - et al.
The Australasian triage scale: examining emergency department nurses’ performance using computer and paper scenarios
Annals of Emergency Medicine
(2004) - et al.
Five-level triage: a report from the ACEP/ENA five-level triage task force
Journal of Emergency Nursing
(2005) Practical Statistics for Medical Research
(1999)- et al.
The importance of nurse caring behaviours as perceived by patients receiving care at an emergency department
Hart and Lung
(2002) - et al.
Implementation Guidelines for the Canadian Emergency Department Triage and Acuity Scale (CTAS)
(1998) Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit
Psychological Bulletin
(1968)- et al.
Does the Manchester triage system detect the critically ill?
Journal of Accident & Emergency Medicine
(1999) - et al.
Poor agreement in application of the Australasian triage scale
Contemporary Nurse
(2003)
Appropriate categorisation of mild pain at triage: a diagnostic study
Emergency Nurse
Predictive validity of a computerizes emergency triage tool
Academic Emergency Medicine
The equivalence of weighted kappa and the intraclass correlation coefficient as measure of reliability
Educational and Psychological Measurement
Consistency of retrospective triage decisions as a standardised instrument for audit
Journal of Accident & Emergency Medicine
Cited by (49)
The new emergency department “Tuscan triage System”. Validation study
2021, International Emergency NursingReliability and validity of three international triage systems within a private health-care group in the Middle East
2020, International Emergency NursingCitation Excerpt :When all the triage categories were put together without any weights applied to the different urgency levels, the agreement between the raters and the reference standard remained moderate. When weights were applied to the urgency levels, either linear or quadratic (e.g., category one vignettes deemed more important than category five’s), the agreements rose significantly to substantial and almost perfect agreement, very similar to what is presented by other studies [16,18,21,26,29,30]. This shows that as the urgency levels increase, the agreement between both the participants and the reference standard also increase.
Setting up standards: A methodological proposal for pediatric Triage machine learning model construction based on clinical outcomes
2019, Expert Systems with ApplicationsTriage Performance in Emergency Medicine: A Systematic Review
2019, Annals of Emergency MedicineValidity of ED: Addressing heterogeneous definitions of over-triage and under-triage
2017, American Journal of Emergency MedicinePerformance of the Manchester Triage System in Adult Medical Emergency Patients: A Prospective Cohort Study
2016, Journal of Emergency MedicineCitation Excerpt :This was particularly true in 20% of patients classified as urgent (Red) and in low urgent categories; most disagreement was found in the Orange category (58.9%). A Swedish study, however, found high interrater reliability and high accuracy of the MTS in the Red and Orange categories, whereas in the less urgent categories accuracy was lower; these findings suggest a possible over-use of resources in low-acuity patients (18). The different findings in these two studies may have arisen because different patient populations had been studied, different methodologies had been used, or limitations might have been inherent in what was regarded as the “gold standard” for accurate assessment of treatment priority.