RESUMEN

Background Large language models (LLMs) for medical applications use unknown amounts of energy, which contribute to the overall carbon footprint of the health care system. Purpose To investigate the tradeoffs between accuracy and energy use when using different LLM types and sizes for medical applications. Materials and Methods This retrospective study evaluated five different billion (B)-parameter sizes of two open-source LLMs (Meta's Llama 2, a general-purpose model, and LMSYS Org's Vicuna 1.5, a specialized fine-tuned model) using chest radiograph reports from the National Library of Medicine's Indiana University Chest X-ray Collection. Reports with missing demographic information and missing or blank files were excluded. Models were run on local compute clusters with visual computing graphic processing units. A single-task prompt explained clinical terminology and instructed each model to confirm the presence or absence of each of the 13 CheXpert disease labels. Energy use (in kilowatt-hours) was measured using an open-source tool. Accuracy was assessed with 13 CheXpert reference standard labels for diagnostic findings on chest radiographs, where overall accuracy was the mean of individual accuracies of all 13 labels. Efficiency ratios (accuracy per kilowatt-hour) were calculated for each model type and size. Results A total of 3665 chest radiograph reports were evaluated. The Vicuna 1.5 7B and 13B models had higher efficiency ratios (737.28 and 331.40, respectively) and higher overall labeling accuracy (93.83% [3438.69 of 3665 reports] and 93.65% [3432.38 of 3665 reports], respectively) than that of the Llama 2 models (7B: efficiency ratio of 13.39, accuracy of 7.91% [289.76 of 3665 reports]; 13B: efficiency ratio of 40.90, accuracy of 74.08% [2715.15 of 3665 reports]; 70B: efficiency ratio of 22.30, accuracy of 92.70% [3397.38 of 3665 reports]). Vicuna 1.5 7B had the highest efficiency ratio (737.28 vs 13.39 for Llama 2 7B). The larger Llama 2 70B model used more than seven times the energy of its 7B counterpart (4.16 kWh vs 0.59 kWh) with low overall accuracy, resulting in an efficiency ratio of only 22.30. Conclusion Smaller fine-tuned LLMs were more sustainable than larger general-purpose LLMs, using less energy without compromising accuracy, highlighting the importance of LLM selection for medical applications. © RSNA, 2024 Supplemental material is available for this article.

Asunto(s)

RESUMEN

Segmentation of infarcts is clinically important in ischemic stroke management and prognostication. It is unclear what role the combination of DWI, ADC, and FLAIR MRI sequences provide for deep learning in infarct segmentation. Recent technologies in model self-configuration have promised greater performance and generalizability through automated optimization. We assessed the utility of DWI, ADC, and FLAIR sequences on ischemic stroke segmentation, compared self-configuring nnU-Net models to conventional U-Net models without manual optimization, and evaluated the generalizability of results on an external clinical dataset. 3D self-configuring nnU-Net models and standard 3D U-Net models with MONAI were trained on 200 infarcts using DWI, ADC, and FLAIR sequences separately and in all combinations. Segmentation results were compared between models using paired t-test comparison on a hold-out test set of 50 cases. The highest performing model was externally validated on a clinical dataset of 50 MRIs. nnU-Net with DWI sequences attained a Dice score of 0.810 ± 0.155. There was no statistically significant difference when DWI sequences were supplemented with ADC and FLAIR images (Dice score of 0.813 ± 0.150; p = 0.15). nnU-Net models significantly outperformed standard U-Net models for all sequence combinations (p < 0.001). On the external dataset, Dice scores measured 0.704 ± 0.199 for positive cases with false positives with intracranial hemorrhage. Highly optimized neural networks such as nnU-Net provide excellent stroke segmentation even when only provided DWI images, without significant improvement from other sequences. This differs from-and significantly outperforms-standard U-Net architectures. Results translated well to the external clinical environment and provide the groundwork for optimized acute stroke segmentation on MRI.

RESUMEN

As the adoption of artificial intelligence (AI) systems in radiology grows, the increase in demand for greater bandwidth and computational resources can lead to greater infrastructural costs for healthcare providers and AI vendors. To that end, we developed ISLE, an intelligent streaming framework to address inefficiencies in current imaging infrastructures. Our framework draws inspiration from video-on-demand platforms to intelligently stream medical images to AI vendors at an optimal resolution for inference from a single high-resolution copy using progressive encoding. We hypothesize that ISLE can dramatically reduce the bandwidth and computational requirements for AI inference, while increasing throughput (i.e., the number of scans processed by the AI system per second). We evaluate our framework by streaming chest X-rays for classification and abdomen CT scans for liver and spleen segmentation and comparing them with the original versions of each dataset. For classification, our results show that ISLE reduced data transmission and decoding time by at least 92% and 88%, respectively, while increasing throughput by more than 3.72 × . For both segmentation tasks, ISLE reduced data transmission and decoding time by at least 82% and 88%, respectively, while increasing throughput by more than 2.9 × . In all three tasks, the ISLE streamed data had no impact on the AI system's diagnostic performance (all P > 0.05). Therefore, our results indicate that our framework can address inefficiencies in current imaging infrastructures by improving data and computational efficiency of AI deployments in the clinical environment without impacting clinical decision-making using AI systems.

RESUMEN

Radiology is a major contributor to health care's impact on climate change, in part due to its reliance on energy-intensive equipment as well as its growing technological reliance. Delivering modern patient care requires a robust informatics team to move images from the imaging equipment to the workstations and the health care system. Radiology informatics is the field that manages medical imaging IT. This involves the acquisition, storage, retrieval, and use of imaging information in health care to improve access and quality, which includes PACS, cloud services, and artificial intelligence. However, the electricity consumption of computing and the life cycle of various computer components expands the carbon footprint of health care. The authors provide a general framework to understand the environmental impact of clinical radiology informatics, which includes using the international Greenhouse Gas Protocol to draft a definition of scopes of emissions pertinent to radiology informatics, as well as exploring existing tools to measure and account for these emissions. A novel standard ecolabel for radiology informatics tools, such as the Energy Star label for consumer devices or Leadership in Energy and Environmental Design certification for buildings, should be developed to promote awareness and guide radiologists and radiology informatics leaders in making environmentally conscious decisions for their clinical practice. At this critical climate juncture, the radiology community has a unique and pressing obligation to consider our shared environmental responsibility in innovating clinical technology for patient care.

Asunto(s)

RESUMEN

See also the editorial by Nikolic in this issue.

Asunto(s)