Manchester Triage in Sweden – Interrater reliability and accuracy

https://doi.org/10.1016/j.ienj.2008.11.008Get rights and content

Abstract

Introduction

This study investigates the interrater reliability and the accuracy of Manchester Triage (MTS) at emergency departments in Western Sweden.

Methods

A group of 79 nurses from seven emergency departments assessed simulated patient cases and assigned triage categories using the same principles as in their daily work. K statistics, accuracy, over-triage and under-triage were then analyzed. The nurses performed 1027 triage assessments.

Results

The result showed an unweighted κ value of 0.61, a linear weighted κ value of 0.71, and a quadratic weighted κ value of 0.81. The determined accuracy was 92% and 91% for the two most urgent categories, but significantly lower for the less urgent categories.

Conclusions

Patients in need of urgent care were identified in more than nine out of 10 cases. The high level of over-triage and under-triage in the less urgent categories resulted in low agreement and accuracy. This may suggest that the resources of emergency departments can be overused for non-urgent patients.

Introduction

Triage, i.e., prioritizing and sorting at emergency departments, has been attracting increasing attention. In particular, older patients with multiple diagnoses are becoming more common (Goodacre et al., 1999, Beveridge et al., 1999, Palmquist and Lindell, 2000, Göransson, 2006). The prioritization carried out has, therefore, also become more complex. More parameters have been introduced for the order in which patients are treated (Baldursdottir and Jonsdottir, 2002, Fernandes et al., 2005, Göransson, 2006).

The triage models, which were developed during the 1990s have been refined and have become national standards in some countries (Fernandes et al., 2005, Worster et al., 2004). Australia was the first to introduce a triage model, “The National Triage Scale”. The model was developed by The Australasian College for Emergency Medicine in 1993. At the beginning of the 21st century its name was changed to “The Australasian Triage Scale” (ATS). In Canada a triage model was developed in the mid-1990s, which was based on the Australian model ATS, “The Canadian Emergency Department Triage and Acuity Scale” (CTAS). In the USA another triage model, “The Emergency Severity Index” (ESI) has been in existence since the end of the 1990s (Beveridge et al., 1998, Gilboy et al., 2005, McCallum Pardey, 2006).

This study focuses on a fifth national standard model, “Manchester Triage” (MTS) (Mackway-Jones, 1997) which is accepted as a standard at emergency departments in Great Britain, Holland and Portugal (Lipley, 2005). In this article we examine the interrater reliability and accuracy using MTS at emergency departments in Western Sweden where the model has been implemented. There have been no earlier studies of the method applicable to Western Sweden. There was, therefore, a need to research the case.

Section snippets

Background

At the beginning of the 21st century discussions began at emergency departments in Western Sweden with regards to implementing a common triage model. At this time most departments used the Swedish National Board of Health and Welfare criteria document which only provided three prioritization levels (Socialstyrelsen, 1994). When the waiting times increased due to an increased patient influx at the emergency departments in Western Sweden, discussions arose that led to the conclusion that the

Method

The research design was a prospective and descriptive survey based on simulated patient cases. The data was collected by asking triage nurses (n = 79) to assess simulated patient cases and determine a triage category according to the same principles used in their daily triage work. The cases had been given reference values by two expert panels. An analysis subsequently took place of the κ values, the accuracy, the over-triage and the under-triage.

Nine emergency departments in Western Sweden use

Results

Seven emergency departments took part in the study. The nurses (n = 79) assessed 13 patient cases each. A total of 1027 triage assessments were analyzed. There were 82% female and 18% male participants, and 91% were older than 25 years of age. The majority of the nurses had worked more than two years after completing their basic training.

The participating nurses from the seven emergency departments together presented an unweighted κ value of 0.61 (SD 95%, CI 0.57–0.65), a linear weighted κ value

Deviation from predicted category

The deviation from the predicted triage categories red and orange was nearly one out of 10. These results would suggest that patients in need of urgent care were identified as such at the emergency departments covered in the study. However, there is still a group of severely ill patients that are not assigned to the predicted categories.

The high percentages of over-triage and under-triage in the yellow and green categories implied that the order of priority was disrupted. It can be significant

Conclusions

In this MTS study we found good interrater reliability at emergency departments in Western Sweden. The accuracy was high (73%). The triage categories red and orange showed the highest values (92% and 91%), which imply that MTS could identify patients in need of early intervention in more than nine out of 10 cases.

This study indicates that the implementation of a structured clinical decision support system could increase the interrater reliability and the accuracy while decreasing over-triage. A

References (30)

  • E. Dann et al.

    Appropriate categorisation of mild pain at triage: a diagnostic study

    Emergency Nurse

    (2005)
  • S.L. Dong et al.

    Predictive validity of a computerizes emergency triage tool

    Academic Emergency Medicine

    (2007)
  • J.L. Fleiss et al.

    The equivalence of weighted kappa and the intraclass correlation coefficient as measure of reliability

    Educational and Psychological Measurement

    (1973)
  • Gilboy, N., Tanabe, P., Travers, D., Rosenau, A., Eitel, D., 2005. Emergency Severity Index, Version 4: Implementation...
  • S.W. Goodacre et al.

    Consistency of retrospective triage decisions as a standardised instrument for audit

    Journal of Accident & Emergency Medicine

    (1999)
  • Cited by (49)

    • Reliability and validity of three international triage systems within a private health-care group in the Middle East

      2020, International Emergency Nursing
      Citation Excerpt :

      When all the triage categories were put together without any weights applied to the different urgency levels, the agreement between the raters and the reference standard remained moderate. When weights were applied to the urgency levels, either linear or quadratic (e.g., category one vignettes deemed more important than category five’s), the agreements rose significantly to substantial and almost perfect agreement, very similar to what is presented by other studies [16,18,21,26,29,30]. This shows that as the urgency levels increase, the agreement between both the participants and the reference standard also increase.

    • Performance of the Manchester Triage System in Adult Medical Emergency Patients: A Prospective Cohort Study

      2016, Journal of Emergency Medicine
      Citation Excerpt :

      This was particularly true in 20% of patients classified as urgent (Red) and in low urgent categories; most disagreement was found in the Orange category (58.9%). A Swedish study, however, found high interrater reliability and high accuracy of the MTS in the Red and Orange categories, whereas in the less urgent categories accuracy was lower; these findings suggest a possible over-use of resources in low-acuity patients (18). The different findings in these two studies may have arisen because different patient populations had been studied, different methodologies had been used, or limitations might have been inherent in what was regarded as the “gold standard” for accurate assessment of treatment priority.

    View all citing articles on Scopus
    View full text