BACKGROUND: In the event of a mass casualty incident (MCI), the situation-related shortage of medical resources does not end when the patients are transported from the scene of the incident. Consequently, an initial triage is required in the receiving hospitals. In the first step, the aim of this study was to create a reference patient vignette set with defined triage categories. This allowed a computer-aided evaluation of the diagnostic quality of triage algorithms for MCI situations in the second step. METHODS: A total of 250 case vignettes validated in practice were entered into a multistage evaluation process by initially 6 and later 36 triage experts. This algorithm-independent expert evaluation of all vignettes-served as the gold standard for analyzing the diagnostic quality of the following triage algorithms: Manchester triage system (MTS module MCI), emergency severity index (ESI), Berlin triage algorithm (BER), the prehospital algorithms PRIOR and mSTaRT, and two project algorithms from a cooperation between the Federal Office of Civil Protection and Disaster Assistance (BBK) and the Hashemite Kingdom of Jordan-intrahospital Jordanian-German project algorithm (JorD) and prehospital triage algorithm (PETRA). Each patient vignette underwent computerized triage through all specified algorithms to obtain comparative test quality outcomes. RESULTS: Of the original 250 vignettes, a triage reference database of 210 patient vignettes was validated independently of the algorithms. These formed the gold standard for comparison of the triage algorithms analyzed. Sensitivities for intrahospital detection of patients in triage category T1 ranged from 1.0 (BER, JorD, PRIOR) to 0.57 (MCI module MTS). Specificities ranged from 0.99 (MTS and PETRA) to 0.67 (PRIOR). Considering Youden's index, BER (0.89) and JorD (0.88) had the best overall performance for detecting patients in triage category T1. Overtriage was most likely with PRIOR, and undertriage with the MCI module of MTS. Up to a decision for category T1, the algorithms require the following numbers of steps given as the median and interquartile range (IQR): ESI 1 (1-2), JorD 1 (1-4), PRIOR 3 (2-4), BER 3 (2-6), mSTaRT 3 (3-5), MTS 4 (4-5) and PETRA 6 (6-8). For the T2 and T3 categories the number of steps until a decision and the test quality of the algorithms are positively interrelated. CONCLUSION: In the present study, transferability of preclinical algorithm-based primary triage results to clinical algorithm-based secondary triage results was demonstrated. The highest diagnostic quality for secondary triage was provided by the Berlin triage algorithm, followed by the Jordanian-German project algorithm for hospitals, which, however, also require the most algorithm steps until a decision.