RESUMEN

Background: The growing demand for genomic testing and limited access to experts necessitate innovative service models. While chatbots have shown promise in supporting genomic services like pre-test counseling, their use in returning positive genetic results, especially using the more recent large language models (LLMs) remains unexplored. Objective: This study reports the prompt engineering process and intrinsic evaluation of the LLM component of a chatbot designed to support returning positive population-wide genomic screening results. Methods: We used a three-step prompt engineering process, including Retrieval-Augmented Generation (RAG) and few-shot techniques to develop an open-response chatbot. This was then evaluated using two hypothetical scenarios, with experts rating its performance using a 5-point Likert scale across eight criteria: tone, clarity, program accuracy, domain accuracy, robustness, efficiency, boundaries, and usability. Results: The chatbot achieved an overall score of 3.88 out of 5 across all criteria and scenarios. The highest ratings were in Tone (4.25), Usability (4.25), and Boundary management (4.0), followed by Efficiency (3.88), Clarity and Robustness (3.81), and Domain Accuracy (3.63). The lowest-rated criterion was Program Accuracy, which scored 3.25. Discussion: The LLM handled open-ended queries and maintained boundaries, while the lower Program Accuracy rating indicates areas for improvement. Future work will focus on refining prompts, expanding evaluations, and exploring optimal hybrid chatbot designs that integrate LLM components with rule-based chatbot components to enhance genomic service delivery.

RESUMEN

Large Language Models (LLMs have the potential to revolutionize clinical medicine by enhancing healthcare access, diagnosis, surgical planning, and education. However, their utilization requires careful, prompt engineering to mitigate challenges like hallucinations and biases. Proper utilization of LLMs involves understanding foundational concepts such as tokenization, embeddings, and attention mechanisms, alongside strategic prompting techniques to ensure accurate outputs. For innovative healthcare solutions, it is essential to maintain ongoing collaboration between AI technology and medical professionals. Ethical considerations, including data security and bias mitigation, are critical to their application. By leveraging LLMs as supplementary resources in research and education, we can enhance learning and support knowledge-based inquiries, ultimately advancing the quality and accessibility of medical care. Continued research and development are necessary to fully realize the potential of LLMs in transforming healthcare.

RESUMEN

The rapid advancements in natural language processing, particularly with the development of Generative Pre-trained Transformer (GPT) models, have opened up new avenues for researchers across various domains. This review article explores the potential of GPT as a research tool, focusing on the core functionalities, key features, and real-world applications of the GPT-4 model. We delve into the concept of prompt engineering, a crucial technique for effectively utilizing GPT, and provide guidelines for designing optimal prompts. Through case studies, we demonstrate how GPT can be applied at various stages of the research process, including literature review, data analysis, and manuscript preparation. The utilization of GPT is expected to enhance research efficiency, stimulate creative thinking, facilitate interdisciplinary collaboration, and increase the impact of research findings. However, it is essential to view GPT as a complementary tool rather than a substitute for human expertise, keeping in mind its limitations and ethical considerations. As GPT continues to evolve, researchers must develop a deep understanding of this technology and leverage its potential to advance their research endeavors while being mindful of its implications.

RESUMEN

Neonatal respiratory distress syndrome (RDS), a severe respiratory illness that is likely to affect preterm newborns especially those who were born preterm with low birth weight (LBW) or multiple births, is one of the complications that preterm babies are likely to develop. Physical Rehabilitation using Oromotor Stimulation, Manual Airway Clearance Technique, Positioning, and Tactile and Kinaesthetic Stimulations (PROMPT) is the intervention followed in this study to determine its effectiveness in the treatment of RDS in LBW triplets. The PROMPT protocol involves interventions such as manually promoting the airway, positioning, oral motor stimulation, and tactile and kinesthetic stimulation. The study examined triplets of similar weight, 1.23g, 1.36g, and 1.18g, at birth. Thus, all known triplets were suffering from the symptoms of RDS like fast breathing and grunting. They were born via premature delivery at 30+5 weeks of pregnancy. Chest X-rays were used as a diagnostic tool for assessing RDS. At the same time, the PROMPT protocol was administered and significant improvements were seen in respiratory health and there was reduced use of mechanical ventilation. The PROMPT protocol shows how effectively an organized method can be applied to treat RDS in LBW triplets.

RESUMEN

PURPOSE: In order to produce a surgical gesture recognition system that can support a wide variety of procedures, either a very large annotated dataset must be acquired, or fitted models must generalize to new labels (so-called zero-shot capability). In this paper we investigate the feasibility of latter option. METHODS: Leveraging the bridge-prompt framework, we prompt-tune a pre-trained vision-text model (CLIP) for gesture recognition in surgical videos. This can utilize extensive outside video data such as text, but also make use of label meta-data and weakly supervised contrastive losses. RESULTS: Our experiments show that prompt-based video encoder outperforms standard encoders in surgical gesture recognition tasks. Notably, it displays strong performance in zero-shot scenarios, where gestures/tasks that were not provided during the encoder training phase are included in the prediction phase. Additionally, we measure the benefit of inclusion text descriptions in the feature extractor training schema. CONCLUSION: Bridge-prompt and similar pre-trained + prompt-tuned video encoder models present significant visual representation for surgical robotics, especially in gesture recognition tasks. Given the diverse range of surgical tasks (gestures), the ability of these models to zero-shot transfer without the need for any task (gesture) specific retraining makes them invaluable.

RESUMEN

Large Language Models (LLM) like ChatGPT-4 hold significant promise in medical application, especially in the field of radiology. While previous studies have shown the promise of ChatGTP-4 in textual-based scenarios, its performance on image-based response remains suboptimal. This study investigates the impact of prompt engineering on ChatGPT-4's accuracy on the 2022 American College of Radiology In Training Test Questions for Diagnostic Radiology Residents that include textual and visual-based questions. Four personas were created, each with unique prompts, and evaluated using ChatGPT-4. Results indicate that encouraging prompts and those disclaiming responsibility led to higher overall accuracy (number of questions answered correctly) compared to other personas. Personas that threaten the LLM with legal action or mounting clinical responsibility were not only found to score less, but also refrain of answering questions at a higher rate. These findings highlight the importance of prompt context in optimizing LLM responses and the need for further research to integrate AI responsibly into medical practice.

RESUMEN

OBJECTIVE: To develop and evaluate a 3D Prompt-ResUNet module that utilized the prompt-based model combined with 3D nnUNet for rapid and consistent autosegmentation of high-risk clinical target volume and organ at risk in high-dose-rate brachytherapy for cervical cancer patients. Approach. We used 73 computed tomography (CT) and 62 magnetic resonance imaging (MRI) scans from 135 (103 for training, 16 for validation, and 16 for testing) cervical cancer patients across two hospitals for HRCTV and OAR segmentation. A novel comparison of the deep learning neural networks 3D Prompt-ResUNet, nnUNet, and SAM-Med3D was applied for the segmentation. Evaluation was conducted in two parts: geometric and clinical assessments. Quantitative metrics included the Dice similarity coefficient (DSC), 95th percentile Hausdorff distance (HD95%), Jaccard index (JI), and Matthews correlation coefficient (MCC). Clinical evaluation involved interobserver comparison, 4-grade expert scoring, and a double-blinded Turing test. Main results. The Prompt-ResUNet model performed most similarly to experienced radiation oncologists, outperforming less experienced ones. During testing, the DSC, HD95% (mm), JI, and MCC value (mean ± SD) for HRCTV were 0.92±0.03, 2.91 ± 0.69, 0.85± 0.04, and 0.92 ± 0.02, respectively. For the bladder, these values were 0.93 ± 0.05, 3.07 ± 1.05, 0.87 ± 0.08, and 0.93 ± 0.05, respectively. For the rectum, they were 0.87 ± 0.03, 3.54 ± 1.46, 0.78 ± 0.05, and 0.87 ± 0.03, respectively. For the sigmoid, they were 0.76 ± 0.11, 7.54 ± 5.54, 0.63 ± 0.14, and 0.78 ± 0.09, respectively. The Prompt-ResUNet achieved a clinical viability score of at least 2 in all evaluation cases (100%) for both HRCTV and bladder and exceeded the 30% positive rate benchmark for all evaluated structures in the Turing test. Significance. The Prompt-ResUNet architecture demonstrated high consistency with ground truth (GT) in autosegmentation of HRCTV and OARs, reducing interobserver variability and shortening treatment times. .

RESUMEN

This study presents an application of an Artificial Neural Network (ANN) to detect fluids in an annular flow regime using Prompt-Gamma Neutron Activation Analysis (PGNAA). The ANN was trained using gamma-ray spectra resulting from neutron interactions with chemical elements found in fluids typical of multiphase flow in oil exploration. These spectra were generated through mathematical simulation using the MCNP6 Monte Carlo computer code to model nuclear particle transport. A241Am-Be polyenergetic neutron source was simulated for these calculations. Several combinations of fluid fractions were developed to create a dataset used for both training and evaluation of the ANN. The ANN demonstrated robust generalization capabilities by accurately predicting the volume fraction of the three investigated fluids (saltwater, oil, and gas), even for cases not included in the training phase. The combination of ANN and PGNAA proved effective for analyzing multiphase systems, with over 92% of all showing errors of less than 5%.

RESUMEN

Diagnostic error is a significant category within preventable patient harm, and it takes many years of effort to develop proficiency in diagnostic reasoning. One of the key challenges medical schools must address is preparing students for the complexity, uncertainty and clinical responsibility in going from student to doctor. Recognising the importance of both cognitive and systems-related factors in diagnostic accuracy, we designed the QUID Prompt (Questions to Use for Improving Diagnosis) for students to refer to at the bedside. This set of questions prompts careful consideration, analysis, and signposting of decision-making processes, to assist students in transitioning from medical school to the real-world of work and achieving diagnostic excellence in clinical settings.

RESUMEN

Accurate prediction of molecular properties is crucial in drug discovery. Traditional methods often overlook that real-world molecules typically exhibit multiple property labels with complex correlations. To this end, we propose a novel framework, HiPM, which stands for Hierarchical Prompted Molecular representation learning framework. HiPM leverages task-aware prompts to enhance the differential expression of tasks in molecular representations and mitigate negative transfer caused by conflicts in individual task information. Our framework comprises two core components: the Molecular Representation Encoder (MRE) and the Task-Aware Prompter (TAP). MRE employs a hierarchical message-passing network architecture to capture molecular features at both the atom and motif levels. Meanwhile, TAP utilizes agglomerative hierarchical clustering algorithm to construct a prompt tree that reflects task affinity and distinctiveness, enabling the model to consider multi-granular correlation information among tasks, thereby effectively handling the complexity of multi-label property prediction. Extensive experiments demonstrate that HiPM achieves state-of-the-art performance across various multi-label datasets, offering a novel perspective on multi-label molecular representation learning.

Asunto(s)

RESUMEN

BACKGROUND: Prompt engineering, focusing on crafting effective prompts to large language models (LLMs), has garnered attention for its capabilities at harnessing the potential of LLMs. This is even more crucial in the medical domain due to its specialized terminology and language technicity. Clinical natural language processing applications must navigate complex language and ensure privacy compliance. Prompt engineering offers a novel approach by designing tailored prompts to guide models in exploiting clinically relevant information from complex medical texts. Despite its promise, the efficacy of prompt engineering in the medical domain remains to be fully explored. OBJECTIVE: The aim of the study is to review research efforts and technical approaches in prompt engineering for medical applications as well as provide an overview of opportunities and challenges for clinical practice. METHODS: Databases indexing the fields of medicine, computer science, and medical informatics were queried in order to identify relevant published papers. Since prompt engineering is an emerging field, preprint databases were also considered. Multiple data were extracted, such as the prompt paradigm, the involved LLMs, the languages of the study, the domain of the topic, the baselines, and several learning, design, and architecture strategies specific to prompt engineering. We include studies that apply prompt engineering-based methods to the medical domain, published between 2022 and 2024, and covering multiple prompt paradigms such as prompt learning (PL), prompt tuning (PT), and prompt design (PD). RESULTS: We included 114 recent prompt engineering studies. Among the 3 prompt paradigms, we have observed that PD is the most prevalent (78 papers). In 12 papers, PD, PL, and PT terms were used interchangeably. While ChatGPT is the most commonly used LLM, we have identified 7 studies using this LLM on a sensitive clinical data set. Chain-of-thought, present in 17 studies, emerges as the most frequent PD technique. While PL and PT papers typically provide a baseline for evaluating prompt-based approaches, 61% (48/78) of the PD studies do not report any nonprompt-related baseline. Finally, we individually examine each of the key prompt engineering-specific information reported across papers and find that many studies neglect to explicitly mention them, posing a challenge for advancing prompt engineering research. CONCLUSIONS: In addition to reporting on trends and the scientific landscape of prompt engineering, we provide reporting guidelines for future studies to help advance research in the medical field. We also disclose tables and figures summarizing medical prompt engineering papers available and hope that future contributions will leverage these existing works to better advance the field.

Asunto(s)

RESUMEN

This study assesses ChatGPT's (GPT-3.5) performance on the 2021 ASPS Plastic Surgery In-Service Examination using prompt modifications and Retrieval Augmented Generation (RAG). ChatGPT was instructed to act as a "resident," "attending," or "medical student," and RAG utilized a curated vector database for context. Results showed no significant improvement, with the "resident" prompt yielding the highest accuracy at 54%, and RAG failing to enhance performance, with accuracy remaining at 54.3%. Despite appropriate reasoning when correct, ChatGPT's overall performance fell in the 10th percentile, indicating the need for fine-tuning and more sophisticated approaches to improve AI's utility in complex medical tasks.

RESUMEN

BACKGROUND AND OBJECTIVE: Biomedical relation extraction aims to reveal the relation between entities in medical texts. Currently, the relation extraction models that have attracted much attention are mainly to fine-tune the pre-trained language models (PLMs) or add template prompt learning, which also limits the ability of the model to deal with grammatical dependencies. Graph convolutional networks (GCNs) can play an important role in processing syntactic dependencies in biomedical texts. METHODS: In this work, we propose a biomedical relation extraction model that fuses GCNs enhanced prompt learning to handle limitations in syntactic dependencies and achieve good performance. Specifically, we propose a model that combines prompt learning with GCNs for relation extraction, by integrating the syntactic dependency information analyzed by GCNs into the prompt learning model, by predicting the correspondence with [MASK] tokens labels for relation extraction. RESULTS: Our model achieved F1 scores of 85.57%, 80.15%, 95.10%, and 84.11% in the biomedical relation extraction datasets GAD, ChemProt, PGR, and DDI, respectively, all of which outperform some existing baseline models. CONCLUSIONS: In this paper, we propose enhancing prompt learning through GCNs, integrating syntactic information into biomedical relation extraction tasks. Experimental results show that our proposed method achieves excellent performance in the biomedical relation extraction task.

Asunto(s)

RESUMEN

Early leaf senescence affects photosynthetic efficiency and limits growth during the late production stage of winter wheat (Triticum aestivum). Natural variation in photosystem response to senescence represents a valuable resource for improving the aging traits of flag leaves. To explore the natural variation of different phases of photosynthetic electron transport in modern wheat cultivars during senescence, we exposed the flag leaves of 32 wheat cultivars to dark conditions to induce senescence process, and simultaneously measured prompt fluorescence and modulated 820 nm reflection. The results showed that the chlorophyll content, activity of PSII donor side, PSI and electron transfer between PSII and PSI were all decreased during dark-induced senescence, but they showed different sensitivity to dark-induced senescence. Furthermore, natural variation in photosynthetic parameters among the 32 wheat cultivars were also observed and showed by variation coefficient of the different parameters. We observed that PSII and PSI activity showed less sensitivity to dark-induced senescence than electron transfer between them, while PSII and PSI activity exhibit greater natural variation than electron transport between PSII and PSI. It suggests that Cytb6f might degrade faster and have less variation than PSII and PSI during dark-induced senescence.

Asunto(s)

RESUMEN

Background: Large language models (LLMs) can assist providers in drafting responses to patient inquiries. We examined a prompt engineering strategy to draft responses for providers in the electronic health record. The aim was to evaluate the change in usability after prompt engineering. Materials and Methods: A pre-post study over 8 months was conducted across 27 providers. The primary outcome was the provider use of LLM-generated messages from Generative Pre-Trained Transformer 4 (GPT-4) in a mixed-effects model, and the secondary outcome was provider sentiment analysis. Results: Of the 7605 messages generated, 17.5% (n = 1327) were used. There was a reduction in negative sentiment with an odds ratio of 0.43 (95% CI, 0.36-0.52), but message use decreased (P < .01). The addition of nurses after the study period led to an increase in message use to 35.8% (P < .01). Discussion: The improvement in sentiment with prompt engineering suggests better content quality, but the initial decrease in usage highlights the need for integration with human factors design. Conclusion: Future studies should explore strategies for optimizing the integration of LLMs into the provider workflow to maximize both usability and effectiveness.

RESUMEN

This study investigated whether the large language model (LLM) utilizes sufficient domain knowledge to reason about critical medical events such as extubation. In detail, we tested whether the LLM accurately comprehends given tabular data and variable importance and whether it can be used in complement to existing ML models such as XGBoost.

Asunto(s)

RESUMEN

Named Entity Recognition (NER) models based on Transformers have gained prominence for their impressive performance in various languages and domains. This work delves into the often-overlooked aspect of entity-level metrics and exposes significant discrepancies between token and entity-level evaluations. The study utilizes a corpus of synthetic French oncological reports annotated with entities representing oncological morphologies. Four different French BERT-based models are fine-tuned for token classification, and their performance is rigorously assessed at both token and entity-level. In addition to fine-tuning, we evaluate ChatGPT's ability to perform NER through prompt engineering techniques. The findings reveal a notable disparity in model effectiveness when transitioning from token to entity-level metrics, highlighting the importance of comprehensive evaluation methodologies in NER tasks. Furthermore, in comparison to BERT, ChatGPT remains limited when it comes to detecting advanced entities in French.

Asunto(s)

RESUMEN

With cancer being a leading cause of death globally, epidemiological and clinical cancer registration is paramount for enhancing oncological care and facilitating scientific research. However, the heterogeneous landscape of medical data presents significant challenges to the current manual process of tumor documentation. This paper explores the potential of Large Language Models (LLMs) for transforming unstructured medical reports into the structured format mandated by the German Basic Oncology Dataset. Our findings indicate that integrating LLMs into existing hospital data management systems or cancer registries can significantly enhance the quality and completeness of cancer data collection - a vital component for diagnosing and treating cancer and improving the effectiveness and benefits of therapies. This work contributes to the broader discussion on the potential of artificial intelligence or LLMs to revolutionize medical data processing and reporting in general and cancer care in particular.

Asunto(s)

RESUMEN

Our study at Chi Mei Medical Center introduced "A+ Nurse," a ChatGPT-based LLM tool, into the nursing documentation process to enhance efficiency and accuracy. The tool offers optimized recording and critical reminders, reducing documentation time from 15 to 5 minutes per patient while maintaining record quality. Nurses appreciated the tool's intuitive design and its effectiveness in improving documentation. This successful integration of AI-generated content in healthcare illustrates the potential of AI to streamline processes and improve patient care, setting a precedent for future AI-driven healthcare innovations.

Asunto(s)

RESUMEN

Open source, lightweight and offline generative large language models (LLMs) hold promise for clinical information extraction due to their suitability to operate in secured environments using commodity hardware without token cost. By creating a simple lupus nephritis (LN) renal histopathology annotation schema and generating gold standard data, this study investigates prompt-based strategies using three state-of-the-art lightweight LLMs, namely BioMistral-DARE-7B (BioMistral), Llama-2-13B (Llama 2), and Mistral-7B-instruct-v0.2 (Mistral). We examine the performance of these LLMs within a zero-shot learning environment for renal histopathology report information extraction. Incorporating four prompting strategies, including combinations of batch prompt (BP), single task prompt (SP), chain of thought (CoT) and standard simple prompt (SSP), our findings indicate that both Mistral and BioMistral consistently demonstrated higher performance compared to Llama 2. Mistral recorded the highest performance, achieving an F1-score of 0.996 [95% CI: 0.993, 0.999] for extracting the numbers of various subtypes of glomeruli across all BP settings and 0.898 [95% CI: 0.871, 0.921] in extracting relational values of immune markers under the BP+SSP setting. This study underscores the capability of offline LLMs to provide accurate and secure clinical information extraction, which can serve as a promising alternative to their heavy-weight online counterparts.

Asunto(s)