RESUMEN
INTRODUCTION: Numerous AI-based systems are being developed to evaluate peripheral blood (PB) smears, but the feasibility of these systems on different smear preparation methods has not been fully understood. In this study, we assessed the impact of different smear preparation methods on the robustness of the deep learning system (DLS). METHODS: We collected 193 PB samples from patients, preparing a pair of smears for each sample using two systems: (1) SP50 smears, prepared by the DLS recommended fully automated slide preparation with double fan drying and staining (May-Grunwald Giemsa, M-G) system using SP50 (Sysmex) and (2) SP1000i smears, prepared by automated smear preparation with single fan drying by SP1000i (Sysmex) and manually stained with M-G. Digital images of PB cells were captured using DI-60 (Sysmex), and the DLS performed cell classification. Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were used to evaluate the performance of the DLS. RESULTS: The specificity and NPV for all cell types were 97.4%-100% in both smear sets. The average sensitivity and PPV were 88.9% and 90.1% on SP50 smears, and 87.0% and 83.2% on SP1000i smears, respectively. The lower performance on SP1000i smears was attributed to the intra-lineage misclassification of neutrophil precursors and inter-lineage misclassification of lymphocytes. CONCLUSION: The DLS demonstrated consistent performance in specificity and NPV for smears prepared by a system different from the recommended method. Our results suggest that applying an automated smear preparation system optimized for the DLS system may be important.
RESUMEN
BACKGROUND: As next-generation sequencing (NGS) oncology tests vary by platform, application, and target of genes, specific methods for external quality assessment (EQA) have not been universally applied. Hence, we have attempted to implement on-site evaluation as EQA in the accreditation program under ISO 15189 for laboratories that perform NGS oncology tests. METHODS: A total of 10 laboratories that performed NGS oncology tests were enrolled. Two types of EQA samples were prepared (Acrometrix Oncology Hotspot Control DNA and OncoSpan gDNA DNA samples), and the variant allele frequency of targeted genes was assigned. The samples were subjected to NGS oncology tests in participant laboratories according to their routine protocols. Based on the result reports, auditors visited the participant laboratories to perform on-site evaluations and provided feedback regarding possible laboratory process improvement. RESULTS: The participant laboratories identified the targeted variants in the Acrometrix Oncology Hotspot Control DNA and OncoSpan gDNA samples with a success rate of 31-100% and 9.5-100%, respectively, compared with reference information, depending on their sequencing systems, and reported a few lower-variant allele frequencies. Six of the eight evaluated laboratories failed to report at least three pathogenic variants due to errors in wet-lab and/or dry-lab processes. Based on the feedback reports and self-assessment, auditors and laboratory staff discussed potential improvements to processes during on-site evaluations for laboratory accreditations. CONCLUSIONS: On-site evaluation as EQA for NGS oncology tests in the laboratory accreditation program under ISO 15189 was successfully implemented and proved applicable to a broad spectrum of NGS tests.