RESUMEN

The multi-lead electrocardiogram (ECG) is widely utilized in clinical diagnosis and monitoring of cardiac conditions. The advancement of deep learning has led to the emergence of automated multi-lead ECG diagnostic networks, which have become essential in the fields of biomedical engineering and clinical cardiac disease diagnosis. Intelligent ECG diagnosis techniques encompass Recurrent Neural Networks (RNN), Transformers, and Convolutional Neural Networks (CNN). While CNN is capable of extracting local spatial information from images, it lacks the ability to learn global spatial features and temporal memory features. Conversely, RNN relies on time and can retain significant sequential features. However, they are not proficient in extracting lengthy dependencies of sequence data in practical scenarios. The self-attention mechanism in the Transformer model has the capability of global feature extraction, but it does not adequately prioritize local features and cannot extract spatial and channel features. This paper proposes STFAC-ECGNet, a model that incorporates CAMV-RNN block, CBMV-CNN block, and TSEF block to enhance the performance of the model by integrating the strengths of CNN, RNN, and Transformer. The CAMV-RNN block incorporates a coordinated adaptive simplified self-attention module that adaptively carries out global sequence feature retention and enhances spatial-temporal information. The CBMV-CNN block integrates spatial and channel attentional mechanism modules in a skip connection, enabling the fusion of spatial and channel information. The TSEF block implements enhanced multi-scale fusion of image spatial and sequence temporal features. In this study, comprehensive experiments were conducted using the PTB-XL large publicly available ECG dataset and the China Physiological Signal Challenge 2018 (CPSC2018) database. The results indicate that STFAC-ECGNet surpasses other cutting-edge techniques in multiple tasks, showcasing robustness and generalization.

Asunto(s)

RESUMEN

Vehicular edge computing (VEC), a promising paradigm for the development of emerging intelligent transportation systems, can provide lower service latency for vehicular applications. However, it is still a challenge to fulfill the requirements of such applications with stringent latency requirements in the VEC system with limited resources. In addition, existing methods focus on handling the offloading task in a certain time slot with statically allocated resources, but ignore the heterogeneous tasks' different resource requirements, resulting in resource wastage. To solve the real-time task offloading and heterogeneous resource allocation problem in VEC system, we propose a decentralized solution based on the attention mechanism and recurrent neural networks (RNN) with a multi-agent distributed deep deterministic policy gradient (AR-MAD4PG). First, to address the partial observability of agents, we construct a shared agent graph and propose a periodic communication mechanism that enables edge nodes to aggregate information from other edge nodes. Second, to help agents better understand the current system state, we design an RNN-based feature extraction network to capture the historical state and resource allocation information of the VEC system. Thirdly, to tackle the challenges of excessive joint observation-action space and ineffective information interference, we adopt the multi-head attention mechanism to compress the dimension of the observation-action space of agents. Finally, we build a simulation model based on the actual vehicle trajectories, and the experimental results show that our proposed method outperforms the existing approaches.

Asunto(s)

RESUMEN

Classification of protein families from their sequences is an enduring task in Proteomics and related studies. Numerous deep-learning models have been moulded to tackle this challenge, but due to the black-box character, they still fall short in reliability. Here, we present a novel explainability pipeline that explains the pivotal decisions of the deep learning model on the classification of the Eukaryotic kinome. Based on a comparative and experimental analysis of the most cutting-edge deep learning algorithms, the best deep learning model CNN-BLSTM was chosen to classify the eight eukaryotic kinase sequences to their corresponding families. As a substitution for the conventional class activation map-based interpretation of CNN-based models in the domain, we have cascaded the GRAD CAM and Integrated Gradient (IG) explainability modus operandi for improved and responsible results. To ensure the trustworthiness of the classifier, we have masked the kinase domain traces, identified from the explainability pipeline and observed a class-specific drop in F1-score from 0.96 to 0.76. In compliance with the Explainable AI paradigm, our results are promising and contribute to enhancing the trustworthiness of deep learning models for biological sequence-associated studies.

Asunto(s)

RESUMEN

Animal behavior emerges from collective dynamics of neurons, making it vulnerable to damage. Paradoxically, many organisms exhibit a remarkable ability to maintain significant behavior even after large-scale neural injury. Molecular underpinnings of this extreme robustness remain largely unknown. Here, we develop a quantitative pipeline to measure long-lasting latent states in planarian flatworm behaviors during whole-brain regeneration. By combining >20,000 animal trials with neural network modeling, we show that long-range volumetric peptidergic signals allow the planarian to rapidly restore coarse behavior output after large perturbations to the nervous system, while slow restoration of small-molecule neuromodulator functions refines precision. This relies on the different time and length scales of neuropeptide and small-molecule transmission to generate incoherent patterns of neural activity that competitively regulate behavior. Controlling behavior through opposing communication mechanisms creates a more robust system than either alone and may serve as a generalizable approach for constructing robust neural networks.

Asunto(s)

RESUMEN

This study aims to explore methods for classifying and describing volleyball training videos using deep learning techniques. By developing an innovative model that integrates Bi-directional Long Short-Term Memory (BiLSTM) and attention mechanisms, referred to BiLSTM-Multimodal Attention Fusion Temporal Classification (BiLSTM-MAFTC), the study enhances the accuracy and efficiency of volleyball video content analysis. Initially, the model encodes features from various modalities into feature vectors, capturing different types of information such as positional and modal data. The BiLSTM network is then used to model multi-modal temporal information, while spatial and channel attention mechanisms are incorporated to form a dual-attention module. This module establishes correlations between different modality features, extracting valuable information from each modality and uncovering complementary information across modalities. Extensive experiments validate the method's effectiveness and state-of-the-art performance. Compared to conventional recurrent neural network algorithms, the model achieves recognition accuracies exceeding 95 % under Top-1 and Top-5 metrics for action recognition, with a recognition speed of 0.04 s per video. The study demonstrates that the model can effectively process and analyze multimodal temporal information, including athlete movements, positional relationships on the court, and ball trajectories. Consequently, precise classification and description of volleyball training videos are achieved. This advancement significantly enhances the efficiency of coaches and athletes in volleyball training and provides valuable insights for broader sports video analysis research.

RESUMEN

Astronomy is entering an unprecedented era of big-data science, driven by missions like the ESA's Gaia telescope, which aims to map the Milky Way in three dimensions. Gaia's vast dataset presents a monumental challenge for traditional analysis methods. The sheer scale of this data exceeds the capabilities of manual exploration, necessitating the utilization of advanced computational techniques. In response to this challenge, we developed a novel approach leveraging deep learning to estimate the metallicity of fundamental mode (ab-type) RR Lyrae stars from their light curves in the Gaia optical G-band. Our study explores applying deep-learning techniques, particularly advanced neural-network architectures, in predicting photometric metallicity from time-series data. Our deep-learning models demonstrated notable predictive performance, with a low mean absolute error (MAE) of 0.0565, the root mean square error (RMSE) of 0.0765, and a high R2 regression performance of 0.9401, measured by cross-validation. The weighted mean absolute error (wMAE) is 0.0563, while the weighted root mean square error (wRMSE) is 0.0763. These results showcase the effectiveness of our approach in accurately estimating metallicity values. Our work underscores the importance of deep learning in astronomical research, particularly with large datasets from missions like Gaia. By harnessing the power of deep-learning methods, we can provide precision in analyzing vast datasets, contributing to more precise and comprehensive insights into complex astronomical phenomena.

RESUMEN

The objective of this study is to investigate methodologies concerning enterprise financial sharing and risk identification to mitigate concerns associated with the sharing and safeguarding of financial data. Initially, the analysis examines security vulnerabilities inherent in conventional financial information sharing practices. Subsequently, blockchain technology is introduced to transition various entity nodes within centralized enterprise financial networks into a decentralized blockchain framework, culminating in the formulation of a blockchain-based model for enterprise financial data sharing. Concurrently, the study integrates the Bi-directional Long Short-Term Memory (BiLSTM) algorithm with the transformer model, presenting an enterprise financial risk identification model referred to as the BiLSTM-fused transformer model. This model amalgamates multimodal sequence modeling with comprehensive understanding of both textual and visual data. It stratifies financial values into levels 1 to 5, where level 1 signifies the most favorable financial condition, followed by relatively good (level 2), average (level 3), high risk (level 4), and severe risk (level 5). Subsequent to model construction, experimental analysis is conducted, revealing that, in comparison to the Byzantine Fault Tolerance (BFT) algorithm mechanism, the proposed model achieves a throughput exceeding 80 with a node count of 146. Both data message leakage and average packet loss rates remain below 10 %. Moreover, when juxtaposed with the recurrent neural networks (RNNs) algorithm, this model demonstrates a risk identification accuracy surpassing 94 %, an AUC value exceeding 0.95, and a reduction in the time required for risk identification by approximately 10 s. Consequently, this study facilitates the more precise and efficient identification of potential risks, thereby furnishing crucial support for enterprise risk management and strategic decision-making endeavors.

RESUMEN

BACKGROUND AND OBJECTIVE: Recent studies point out that the dynamics and interaction of cell populations within their environment are related to several biological processes in immunology. Hence, single-cell analysis in immunology now relies on spatial omics. Moreover, recent literature suggests that immunology scenarios are hierarchically organized, including unknown cell behaviors appearing in different proportions across some observable control and therapy groups. These dynamic behaviors play a crucial role in identifying the causes of processes such as inflammation, aging, and fighting off pathogens or cancerous cells. In this work, we use a self-supervised learning approach to discover these behaviors associated with cell dynamics in an immunology scenario. MATERIALS AND METHODS: Specifically, we study the different responses of control group and therapy groups in a scenario involving inflammation due to infarct, with a focus on neutrophil migration within blood vessels. Starting from a set of hand-crafted spatio-temporal features, we use a recurrent neural network to generate embeddings that properly describe the dynamics of the migration processes. The network is trained using a novel multi-task contrastive loss that, on the one hand, models the hierarchical structure of our scenario (groups-behaviors-samples) and, on the other, ensures temporal consistency within the embedding, enforcing that subsequent temporal samples obtained from a given cell stay close in the latent space. RESULTS: Our experimental results demonstrate that the resulting embeddings improve the separability of cell behaviors and log-likelihood of the therapies, when compared to the hand-crafted feature extraction and recent methods from the state of the art, even with dimensionality reduction (16 vs. 21 hand-crafted features). CONCLUSIONS: Our approach enables single-cell analyses at a population level, being able to automatically discover shared behaviors among different groups. This, in turn, enables the prediction of the therapy effectiveness based on their proportions within a study group.

Asunto(s)

RESUMEN

A stimulus held in working memory is perceived as contracted toward the average stimulus. This contraction bias has been extensively studied in psychophysics, but little is known about its origin from neural activity. By training recurrent networks of spiking neurons to discriminate temporal intervals, we explored the causes of this bias and how behavior relates to population firing activity. We found that the trained networks exhibited animal-like behavior. Various geometric features of neural trajectories in state space encoded warped representations of the durations of the first interval modulated by sensory history. Formulating a normative model, we showed that these representations conveyed a Bayesian estimate of the interval durations, thus relating activity and behavior. Importantly, our findings demonstrate that Bayesian computations already occur during the sensory phase of the first stimulus and persist throughout its maintenance in working memory, until the time of stimulus comparison.

Asunto(s)

RESUMEN

Handwritten Text Recognition (HTR) is a challenging task due to the complex structures and variations present in handwritten text. In recent years, the application of gated mechanisms, such as Long Short-Term Memory (LSTM) networks, has brought significant advancements to HTR systems. This paper presents an overview of HTR using a gated mechanism and highlights its novelty and advantages. The gated mechanism enables the model to capture long-term dependencies, retain relevant context, handle variable length sequences, mitigate error propagation, and adapt to contextual variations. The pipeline involves preprocessing the handwritten text images, extracting features, modeling the sequential dependencies using the gated mechanism, and decoding the output into readable text. The training process utilizes annotated datasets and optimization techniques to minimize transcription discrepancies. HTR using a gated mechanism has found applications in digitizing historical documents, automatic form processing, and real-time transcription. The results show improved accuracy and robustness compared to traditional HTR approaches. The advancements in HTR using a gated mechanism open up new possibilities for effectively recognizing and transcribing handwritten text in various domains. This research does a better job than the most recent iteration of the HTR system when compared to five different handwritten datasets (Washington, Saint Gall, RIMES, Bentham and IAM). Smartphones and robots are examples of low-cost computing devices that can benefit from this research.

RESUMEN

Recurrent Neural Networks (RNNs), a type of machine learning technique, have recently drawn a lot of interest in numerous fields, including epidemiology. Implementing public health interventions in the field of epidemiology depends on efficient modeling and outbreak prediction. Because RNNs can capture sequential dependencies in data, they have become highly effective tools in this field. In this paper, the use of RNNs in epidemic modeling is examined, with a focus on the extent to which they can handle the inherent temporal dynamics in the spread of diseases. The mathematical representation of epidemics requires taking time-dependent variables into account, such as the rate at which infections spread and the long-term effects of interventions. The goal of this study is to use an intelligent computing solution based on RNNs to provide numerical performances and interpretations for the SEIR nonlinear system based on the propagation of the Zika virus (SEIRS-PZV) model. The four patient dynamics, namely susceptible patients S(y), exposed patients admitted in a hospital E(y), the fraction of infective individuals I(y), and recovered patients R(y), are represented by the epidemic version of the nonlinear system, or the SEIR model. SEIRS-PZV is represented by ordinary differential equations (ODEs), which are then solved by the Adams method using the Mathematica software to generate a dataset. The dataset was used as an output for the RNN to train the model and examine results such as regressions, correlations, error histograms, etc. For RNN, we used 100% to train the model with 15 hidden layers and a delay of 2 seconds. The input for the RNN is a time series sequence from 0 to 5, with a step size of 0.05. In the end, we compared the approximated solution with the exact solution by plotting them on the same graph and generating the absolute error plot for each of the 4 cases of SEIRS-PZV. Predictions made by the model appeared to be become more accurate when the mean squared error (MSE) decreased. An increased fit to the observed data was suggested by this decrease in the MSE, which suggested that the variance between the model's predicted values and the actual values was dropping. A minimal absolute error almost equal to zero was obtained, which further supports the usefulness of the suggested strategy. A small absolute error shows the degree to which the model's predictions matches the ground truth values, thus indicating the level of accuracy and precision for the model's output.

RESUMEN

How do biological neural systems efficiently encode, transform and propagate information between the sensory periphery and the sensory cortex about sensory features evolving at different time scales? Are these computations efficient in normative information processing terms? While previous work has suggested that biologically plausible models of of such neural information processing may be implemented efficiently within a single processing layer, how such computations extend across several processing layers is less clear. Here, we model propagation of multiple time-varying sensory features across a sensory pathway, by extending the theory of efficient coding with spikes to efficient encoding, transformation and transmission of sensory signals. These computations are optimally realized by a multilayer spiking network with feedforward networks of spiking neurons (receptor layer) and recurrent excitatory-inhibitory networks of generalized leaky integrate-and-fire neurons (recurrent layers). Our model efficiently realizes a broad class of feature transformations, including positive and negative interaction across features, through specific and biologically plausible structures of feedforward connectivity. We find that mixing of sensory features in the activity of single neurons is beneficial because it lowers the metabolic cost at the network level. We apply the model to the somatosensory pathway by constraining it with parameters measured empirically and include in its last node, analogous to the primary somatosensory cortex (S1), two types of inhibitory neurons: parvalbumin-positive neurons realizing lateral inhibition, and somatostatin-positive neurons realizing winner-take-all inhibition. By implementing a negative interaction across stimulus features, this model captures several intriguing empirical observations from the somatosensory system of the mouse, including a decrease of sustained responses from subcortical networks to S1, a non-linear effect of the knock-out of receptor neuron types on the activity in S1, and amplification of weak signals from sensory neurons across the pathway.

RESUMEN

Models have always been central to inferring molecular evolution and to reconstructing phylogenetic trees. Their use typically involves the development of a mechanistic framework reflecting our understanding of the underlying biological processes, such as nucleotide substitu- tions, and the estimation of model parameters by maximum likelihood or Bayesian inference. However, deriving and optimizing the likelihood of the data is not always possible under complex evolutionary scenarios or even tractable for large datasets, often leading to unrealistic simplifying assumptions in the fitted models. To overcome this issue, we coupled stochastic simulations of genome evolution with a new supervised deep learning model to infer key parameters of molecular evolution. Our model is designed to directly analyze multiple sequence alignments and estimate per-site evolutionary rates and divergence, without requiring a known phylogenetic tree. The accuracy of our predictions matched that of likelihood-based phylogenetic inference, when rate heterogeneity followed a simple gamma distribution, but it strongly exceeded it under more complex patterns of rate variation, such as codon models. Our approach is highly scalable and can be efficiently applied to genomic data, as we showed on a dataset of 26 million nucleotides from the clownfish clade. Our simulations also showed that the integration of per-site rates obtained by deep learning within a Bayesian framework led to significantly more accu- rate phylogenetic inference, particularly with respect to the estimated branch lengths. We thus propose that future advancements in phylogenetic analysis will benefit from a semi-supervised learning approach that combines deep-learning estimation of substitution rates, which allows for more flexible models of rate variation, and probabilistic inference of the phylogenetic tree, which guarantees interpretability and a rigorous assessment of statistical support.

RESUMEN

Severe acute respiratory syndrome coronavirus 2 (SARS­CoV­2) has been evolving rapidly after causing havoc worldwide in 2020. Since then, it has been very hard to contain the virus owing to its frequently mutating nature. Changes in its genome lead to viral evolution, rendering it more resistant to existing vaccines and drugs. Predicting viral mutations beforehand will help in gearing up against more infectious and virulent versions of the virus in turn decreasing the damage caused by them. In this paper, we have proposed different NMT (neural machine translation) architectures based on RNNs (recurrent neural networks) to predict mutations in the SARS-CoV-2-selected non-structural proteins (NSP), i.e., NSP1, NSP3, NSP5, NSP8, NSP9, NSP13, and NSP15. First, we created and pre-processed the pairs of sequences from two languages using k-means clustering and nearest neighbors for training a neural translation machine. We also provided insights for training NMTs on long biological sequences. In addition, we evaluated and benchmarked our models to demonstrate their efficiency and reliability.

Asunto(s)

RESUMEN

Recurrent neural networks (RNNs) transmit information over time through recurrent connections. In contrast, biological neural networks use many other temporal processing mechanisms. One of these mechanisms is the inter-neuron delays caused by varying axon properties. Recently, this feature was implemented in echo state networks (ESNs), a type of RNN, by assigning spatial locations to neurons and introducing distance-dependent inter-neuron delays. These delays were shown to significantly improve ESN task performance. However, thus far, it is still unclear why distance-based delay networks (DDNs) perform better than ESNs. In this paper, we show that by optimizing inter-node delays, the memory capacity of the network matches the memory requirements of the task. As such, networks concentrate their memory capabilities to the points in the past which contain the most information for the task at hand. Moreover, we show that DDNs have a greater total linear memory capacity, with the same amount of non-linear processing power.

RESUMEN

Oral cancer early diagnosis is a critical task in the field of medical science, and one of the most necessary things is to develop sound and effective strategies for early detection. The current research investigates a new strategy to diagnose an oral cancer based upon combination of effective learning and medical imaging. The current research investigates a new strategy to diagnose an oral cancer using Gated Recurrent Unit (GRU) networks optimized by an improved model of the NGO (Northern Goshawk Optimization) algorithm. The proposed approach has several advantages over existing methods, including its ability to analyze large and complex datasets, its high accuracy, as well as its capacity to detect oral cancer at the very beginning stage. The improved NGO algorithm is utilized to improve the GRU network that helps to improve the performance of the network and increase the accuracy of the diagnosis. The paper describes the proposed approach and evaluates its performance using a dataset of oral cancer patients. The findings of the study demonstrate the efficiency of the suggested approach in accurately diagnosing oral cancer.

RESUMEN

In order to comprehend and enhance models that describes various brain regions it is important to study the dynamics of trained recurrent neural networks. Including Dale's law in such models usually presents several challenges. However, this is an important aspect that allows computational models to better capture the characteristics of the brain. Here we present a framework to train networks using such constraint. Then we have used it to train them in simple decision making tasks. We characterized the eigenvalue distributions of the recurrent weight matrices of such networks. Interestingly, we discovered that the non-dominant eigenvalues of the recurrent weight matrix are distributed in a circle with a radius less than 1 for those whose initial condition before training was random normal and in a ring for those whose initial condition was random orthogonal. In both cases, the radius does not depend on the fraction of excitatory and inhibitory units nor the size of the network. Diminution of the radius, compared to networks trained without the constraint, has implications on the activity and dynamics that we discussed here. Supplementary Information: The online version contains supplementary material available at 10.1007/s11571-023-09956-w.

RESUMEN

Research on transformers in remote sensing (RS), which started to increase after 2021, is facing the problem of a relative lack of review. To understand the trends of transformers in RS, we undertook a quantitative analysis of the major research on transformers over the past two years by dividing the application of transformers into eight domains: land use/land cover (LULC) classification, segmentation, fusion, change detection, object detection, object recognition, registration, and others. Quantitative results show that transformers achieve a higher accuracy in LULC classification and fusion, with more stable performance in segmentation and object detection. Combining the analysis results on LULC classification and segmentation, we have found that transformers need more parameters than convolutional neural networks (CNNs). Additionally, further research is also needed regarding inference speed to improve transformers' performance. It was determined that the most common application scenes for transformers in our database are urban, farmland, and water bodies. We also found that transformers are employed in the natural sciences such as agriculture and environmental protection rather than the humanities or economics. Finally, this work summarizes the analysis results of transformers in remote sensing obtained during the research process and provides a perspective on future directions of development.

RESUMEN

In the paper, a new evolutionary technique called Linear Matrix Genetic Programming (LMGP) is proposed. It is a matrix extension of Linear Genetic Programming and its application is data-driven black-box control-oriented modeling in conditions of limited access to training data. In LMGP, the model is in the form of an evolutionarily-shaped program which is a sequence of matrix operations. Since the program has a hidden state, running it for a sequence of input data has a similar effect to using well-known recurrent neural networks such as Long Short-Term Memory (LSTM) or Gated Recurrent Unit (GRU). To verify the effectiveness of the LMGP, it was compared with different types of neural networks. The task of all the compared techniques was to reproduce the behavior of a nonlinear model of an underwater vehicle. The results of the comparative tests are reported in the paper and they show that the LMGP can quickly find an effective and very simple solution to the given problem. Moreover, a detailed comparison of models, generated by LMGP and LSTM/GRU, revealed that the former are up to four times more accurate than the latter in reproducing vehicle behavior.

RESUMEN

INTRODUCTION: Ecological Momentary Assessment (EMA) holds promise for providing insights into daily life experiences when studying mental health phenomena. However, commonly used mixed-effects linear statistical models do not fully utilize the richness of the ultidimensional time-varying data that EMA yields. Recurrent Neural Networks (RNNs) provide an alternative data analytic method to leverage more information and potentially improve prediction, particularly for non-normally distributed outcomes. METHODS: As part of a broader research study of suicidal thoughts and behavior in people with borderline personality disorder (BPD), eighty-four participants engaged in EMA data collection over one week, answering questions multiple times each day about suicidal ideation (SI), stressful events, coping strategy use, and affect. RNNs and mixed-effects linear regression models (MEMs) were trained and used to predict SI. Root mean squared error (RMSE), mean absolute percent error (MAPE), and a pseudo-R2 accuracy metric were used to compare SI prediction accuracy between the two modeling methods. RESULTS: RNNs had superior accuracy metrics (full model: RMSE = 3.41, MAPE = 42 %, pseudo-R2 = 26 %) compared with MEMs (full model: RMSE = 3.84, MAPE = 56 %, pseudo-R2 = 16 %). Importantly, RNNs showed significantly more accurate prediction at higher values of SI. Additionally, RNNs predicted, with significantly higher accuracy, the SI scores of participants with depression diagnoses and of participants with higher depression scores at baseline. CONCLUSION: In this EMA study with a moderately sized sample, RNNs were better able to learn and predict daily SI compared with mixed-effects models. RNNs should be considered as an option for EMA analysis.

Asunto(s)