Your browser doesn't support javascript.
loading
Combining human and machine intelligence for clinical trial eligibility querying.
Fang, Yilu; Idnay, Betina; Sun, Yingcheng; Liu, Hao; Chen, Zhehuan; Marder, Karen; Xu, Hua; Schnall, Rebecca; Weng, Chunhua.
Afiliación
  • Fang Y; Department of Biomedical Informatics, Columbia University, New York, New York, USA.
  • Idnay B; School of Nursing, Columbia University, New York, New York, USA.
  • Sun Y; Department of Neurology, Columbia University, New York, New York, USA.
  • Liu H; Department of Biomedical Informatics, Columbia University, New York, New York, USA.
  • Chen Z; Department of Biomedical Informatics, Columbia University, New York, New York, USA.
  • Marder K; Department of Biomedical Informatics, Columbia University, New York, New York, USA.
  • Xu H; Department of Neurology, Columbia University, New York, New York, USA.
  • Schnall R; School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, Texas, USA.
  • Weng C; School of Nursing, Columbia University, New York, New York, USA.
J Am Med Inform Assoc ; 29(7): 1161-1171, 2022 06 14.
Article en En | MEDLINE | ID: mdl-35426943
OBJECTIVE: To combine machine efficiency and human intelligence for converting complex clinical trial eligibility criteria text into cohort queries. MATERIALS AND METHODS: Criteria2Query (C2Q) 2.0 was developed to enable real-time user intervention for criteria selection and simplification, parsing error correction, and concept mapping. The accuracy, precision, recall, and F1 score of enhanced modules for negation scope detection, temporal and value normalization were evaluated using a previously curated gold standard, the annotated eligibility criteria of 1010 COVID-19 clinical trials. The usability and usefulness were evaluated by 10 research coordinators in a task-oriented usability evaluation using 5 Alzheimer's disease trials. Data were collected by user interaction logging, a demographic questionnaire, the Health Information Technology Usability Evaluation Scale (Health-ITUES), and a feature-specific questionnaire. RESULTS: The accuracies of negation scope detection, temporal and value normalization were 0.924, 0.916, and 0.966, respectively. C2Q 2.0 achieved a moderate usability score (3.84 out of 5) and a high learnability score (4.54 out of 5). On average, 9.9 modifications were made for a clinical study. Experienced researchers made more modifications than novice researchers. The most frequent modification was deletion (5.35 per study). Furthermore, the evaluators favored cohort queries resulting from modifications (score 4.1 out of 5) and the user engagement features (score 4.3 out of 5). DISCUSSION AND CONCLUSION: Features to engage domain experts and to overcome the limitations in automated machine output are shown to be useful and user-friendly. We concluded that human-computer collaboration is key to improving the adoption and user-friendliness of natural language processing.
Asunto(s)
Palabras clave
Texto completo
Añadir a Mi BVS
Imprimir
XML
PubMed Links
Buscar en Google

Texto completo: 1 Colección: 01-internacional Base de datos: MEDLINE Asunto principal: COVID-19 Tipo de estudio: Prognostic_studies Límite: Humans Idioma: En Revista: J Am Med Inform Assoc Asunto de la revista: INFORMATICA MEDICA Año: 2022 Tipo del documento: Article País de afiliación: Estados Unidos Pais de publicación: Reino Unido
Texto completo
Añadir a Mi BVS
Imprimir
XML
PubMed Links
Buscar en Google

Texto completo: 1 Colección: 01-internacional Base de datos: MEDLINE Asunto principal: COVID-19 Tipo de estudio: Prognostic_studies Límite: Humans Idioma: En Revista: J Am Med Inform Assoc Asunto de la revista: INFORMATICA MEDICA Año: 2022 Tipo del documento: Article País de afiliación: Estados Unidos Pais de publicación: Reino Unido