Objective: To measure and compare the reliability and predictive validity of a four-level triage system (I-4L) and the new four-level model triage emergency method (TEM).
Methods: This observational study was conducted in an urban hospital. Ten nurses were randomly selected to assign a triage level to 189 paper scenarios, using either the I-4L model (5 nurses) or the TEM model (5 nurses). We used weighted kappa statistics to measure the interrater and intrarater reliability of each triage tool and assessed the validity of each models based on the accuracy in predicting admission.
Results: Interrater reliability was kappa=0.73 [95% CI (confidence interval): 0.59-0.87] and kappa=0.79 (95% CI: 0.65-0.93) with I-4L and TEM, respectively. Intrarater reliability was kappa=0.82 (95% CI: 0.67-0.96) and kappa=0.78 (95% CI: 0.62-0.93), respectively. The accuracy of triage rating for admission prediction was similarly good with I-4L and TEM, namely, 79% (95% CI: 74-85) and 77% (95% CI: 74-85). The proportion of patients admitted per triage level was similar with the two models.
Conclusion: The interrater and intrarater reliability for rating triage acuity and for accuracy in patient admission prediction was good with both models. Performance with the new model was similar to that of I-4L despite the nurses' short experience. The new TEM model has the advantage of predicting utilization of emergency department resources.