RESUMEN

The misdiagnosis of headache disorders is a serious issue, and AI-based headache model diagnoses with external validation are scarce. We previously developed an artificial intelligence (AI)-based headache diagnosis model using a database of 4000 patients' questionnaires in a headache-specializing clinic and herein performed external validation prospectively. The validation cohort of 59 headache patients was prospectively collected from August 2023 to February 2024 at our or collaborating multicenter institutions. The ground truth was specialists' diagnoses based on the initial questionnaire and at least a one-month headache diary after the initial consultation. The diagnostic performance of the AI model was evaluated. The mean age was 42.55 ± 12.74 years, and 51/59 (86.67%) of the patients were female. No missing values were reported. Of the 59 patients, 56 (89.83%) had migraines or medication-overuse headaches, and 3 (5.08%) had tension-type headaches. No one had trigeminal autonomic cephalalgias or other headaches. The models' overall accuracy and kappa for the ground truth were 94.92% and 0.65 (95%CI 0.21-1.00), respectively. The sensitivity, specificity, precision, and F values for migraines were 98.21%, 66.67%, 98.21%, and 98.21%, respectively. There was disagreement between the AI diagnosis and the ground truth by headache specialists in two patients. This is the first external validation of the AI headache diagnosis model. Further data collection and external validation are required to strengthen and improve its performance in real-world settings.

RESUMEN

OBJECTIVE: We developed an artificial intelligence (AI)-based headache diagnosis model using a large questionnaire database from a headache-specializing clinic. BACKGROUND: Misdiagnosis of headache disorders is a serious issue and AI-based headache diagnosis models are scarce. METHODS: We developed an AI-based headache diagnosis model and conducted internal validation based on a retrospective investigation of 6058 patients (4240 training dataset for model development and 1818 test dataset for internal validation) diagnosed by a headache specialist. The ground truth was the diagnosis by the headache specialist. The diagnostic performance of the AI model was evaluated. RESULTS: The dataset included 4829/6058 (79.7%) patients with migraine, 834/6058 (13.8%) with tension-type headache, 78/6058 (1.3%) with trigeminal autonomic cephalalgias, 38/6058 (0.6%) with other primary headache disorders, and 279/6058 (4.6%) with other headaches. The mean (standard deviation) age was 34.7 (14.5) years, and 3986/6058 (65.8%) were female. The model's micro-average accuracy, sensitivity (recall), specificity, precision, and F-values for the test dataset were 93.7%, 84.2%, 84.2%, 96.1%, and 84.2%, respectively. The diagnostic performance for migraine was high, with a sensitivity of 88.8% and c-statistics of 0.89 (95% confidence interval 0.87-0.91). CONCLUSIONS: Our AI model demonstrated high diagnostic performance for migraine. If secondary headaches can be ruled out, the model can be a powerful tool for diagnosing migraine; however, further data collection and external validation are required to strengthen the performance, ensure the generalizability in other outpatients, and demonstrate its utility in real-world settings.

Asunto(s)

Trastornos Migrañosos , Cefalea de Tipo Tensional , Humanos , Femenino , Adulto , Masculino , Inteligencia Artificial , Estudios Retrospectivos , Cefalea/diagnóstico , Trastornos Migrañosos/diagnóstico

RESUMEN

Introduction Raising stroke awareness is important to shorten the interval from onset to consultation. We performed a school-based stroke education by on-demand e-learning during the coronavirus disease 2019 pandemic. Methods We performed on-demand e-learning and distributed the online- and paper-based manga about stroke for students and parental guardians in August 2021. We carried out this in a manner similar to the prior effective online stroke awareness initiatives in Japan. An online post-educational survey in October 2021 was conducted to evaluate the awareness effects by asking participants about their knowledge. We also investigated the modified Rankin Scale (mRS) at the discharge of stroke patients who were treated in our hospital during the before- and after-campaign periods, respectively. Results We distributed the paper-based manga and asked to work on this campaign to all 2,429 students (1,545 elementary school and 884 junior high school students) who lived in Itoigawa. We acquired 261 (10.7%) online responses from the students and 211 (8.7%) responses from their parental guardians. The number of students who chose all correct answers in the survey significantly increased after the campaign (205/261, 78.5%) compared to that before the campaign (135/261, 51.7%) and those of parental guardians showed similar trends (before campaign 93/211, 44.1%; after campaign 198/211, 93.8%). We investigated 282 stroke patients (90 patients before and 192 patients after-campaign period), and their mRS at discharge after-campaign seemed to be improved. Conclusion Only 10.7% of students and 8.7% of the parental guardians worked on the online survey. However, the number of those who chose correct answers about stroke increased after the campaign. After this campaign, the mRS of stroke patients at discharge improved although it was unclear if this is a direct result of this activity.