Your browser doesn't support javascript.
loading
Measuring cognitive effort using tabular transformer-based language models of electronic health record-based audit log action sequences.
Kim, Seunghwan; Warner, Benjamin C; Lew, Daphne; Lou, Sunny S; Kannampallil, Thomas.
Afiliación
  • Kim S; Roy and Diana Vagelos Division of Biology and Biomedical Sciences, Washington University in St. Louis, St. Louis, MO 63110, United States.
  • Warner BC; Institute for Informatics, Data Science and Biostatistics (I2DB), Washington University School of Medicine, St. Louis, MO 63110, United States.
  • Lew D; Department of Computer Science and Engineering, Washington University St. Louis, St. Louis, MO 63130-4899, United States.
  • Lou SS; Institute for Informatics, Data Science and Biostatistics (I2DB), Washington University School of Medicine, St. Louis, MO 63110, United States.
  • Kannampallil T; Institute for Informatics, Data Science and Biostatistics (I2DB), Washington University School of Medicine, St. Louis, MO 63110, United States.
J Am Med Inform Assoc ; 31(10): 2228-2235, 2024 Oct 01.
Article en En | MEDLINE | ID: mdl-39001791
ABSTRACT

OBJECTIVES:

To develop and validate a novel measure, action entropy, for assessing the cognitive effort associated with electronic health record (EHR)-based work activities. MATERIALS AND

METHODS:

EHR-based audit logs of attending physicians and advanced practice providers (APPs) from four surgical intensive care units in 2019 were included. Neural language models (LMs) were trained and validated separately for attendings' and APPs' action sequences. Action entropy was calculated as the cross-entropy associated with the predicted probability of the next action, based on prior actions. To validate the measure, a matched pairs study was conducted to assess the difference in action entropy during known high cognitive effort scenarios, namely, attention switching between patients and to or from the EHR inbox.

RESULTS:

Sixty-five clinicians performing 5 904 429 EHR-based audit log actions on 8956 unique patients were included. All attention switching scenarios were associated with a higher action entropy compared to non-switching scenarios (P < .001), except for the from-inbox switching scenario among APPs. The highest difference among attendings was for the from-inbox attention switching Action entropy was 1.288 (95% CI, 1.256-1.320) standard deviations (SDs) higher for switching compared to non-switching scenarios. For APPs, the highest difference was for the to-inbox switching, where action entropy was 2.354 (95% CI, 2.311-2.397) SDs higher for switching compared to non-switching scenarios.

DISCUSSION:

We developed a LM-based metric, action entropy, for assessing cognitive burden associated with EHR-based actions. The metric showed discriminant validity and statistical significance when evaluated against known situations of high cognitive effort (ie, attention switching). With additional validation, this metric can potentially be used as a screening tool for assessing behavioral action phenotypes that are associated with higher cognitive burden.

CONCLUSION:

An LM-based action entropy metric-relying on sequences of EHR actions-offers opportunities for assessing cognitive effort in EHR-based workflows.
Asunto(s)
Palabras clave
Texto completo
Añadir a Mi BVS
Imprimir
XML
PubMed Links
Buscar en Google

Texto completo: 1 Colección: 01-internacional Base de datos: MEDLINE Asunto principal: Cognición / Registros Electrónicos de Salud Límite: Humans Idioma: En Revista: J Am Med Inform Assoc Asunto de la revista: INFORMATICA MEDICA Año: 2024 Tipo del documento: Article País de afiliación: Estados Unidos Pais de publicación: Reino Unido
Texto completo
Añadir a Mi BVS
Imprimir
XML
PubMed Links
Buscar en Google

Texto completo: 1 Colección: 01-internacional Base de datos: MEDLINE Asunto principal: Cognición / Registros Electrónicos de Salud Límite: Humans Idioma: En Revista: J Am Med Inform Assoc Asunto de la revista: INFORMATICA MEDICA Año: 2024 Tipo del documento: Article País de afiliación: Estados Unidos Pais de publicación: Reino Unido