RESUMEN

Objective. Vagus nerve stimulation (VNS) is being investigated as a potential therapy for cardiovascular diseases including heart failure, cardiac arrhythmia, and hypertension. The lack of a systematic approach for controlling and tuning the VNS parameters poses a significant challenge. Closed-loop VNS strategies combined with artificial intelligence (AI) approaches offer a framework for systematically learning and adapting the optimal stimulation parameters. In this study, we presented an interactive AI framework using reinforcement learning (RL) for automated data-driven design of closed-loop VNS control systems in a computational study.Approach.Multiple simulation environments with a standard application programming interface were developed to facilitate the design and evaluation of the automated data-driven closed-loop VNS control systems. These environments simulate the hemodynamic response to multi-location VNS using biophysics-based computational models of healthy and hypertensive rat cardiovascular systems in resting and exercise states. We designed and implemented the RL-based closed-loop VNS control frameworks in the context of controlling the heart rate and the mean arterial pressure for a set point tracking task. Our experimental design included two approaches; a general policy using deep RL algorithms and a sample-efficient adaptive policy using probabilistic inference for learning and control.Main results.Our simulation results demonstrated the capabilities of the closed-loop RL-based approaches to learn optimal VNS control policies and to adapt to variations in the target set points and the underlying dynamics of the cardiovascular system. Our findings highlighted the trade-off between sample-efficiency and generalizability, providing insights for proper algorithm selection. Finally, we demonstrated that transfer learning improves the sample efficiency of deep RL algorithms allowing the development of more efficient and personalized closed-loop VNS systems.Significance.We demonstrated the capability of RL-based closed-loop VNS systems. Our approach provided a systematic adaptable framework for learning control strategies without requiring prior knowledge about the underlying dynamics.

Asunto(s)

Simulación por Computador , Refuerzo en Psicología , Estimulación del Nervio Vago , Estimulación del Nervio Vago/métodos , Animales , Ratas , Frecuencia Cardíaca/fisiología , Sistema Cardiovascular , Algoritmos , Inteligencia Artificial

RESUMEN

Deep Brain Stimulation (DBS) stands as an effective intervention for alleviating the motor symptoms of Parkinson's disease (PD). Traditional commercial DBS devices are only able to deliver fixed-frequency periodic pulses to the basal ganglia (BG) regions of the brain, i.e., continuous DBS (cDBS). However, they in general suffer from energy inefficiency and side effects, such as speech impairment. Recent research has focused on adaptive DBS (aDBS) to resolve the limitations of cDBS. Specifically, reinforcement learning (RL) based approaches have been developed to adapt the frequencies of the stimuli in order to achieve both energy efficiency and treatment efficacy. However, RL approaches in general require significant amount of training data and computational resources, making it intractable to integrate RL policies into real-time embedded systems as needed in aDBS. In contrast, contextual multi-armed bandits (CMAB) in general lead to better sample efficiency compared to RL. In this study, we propose a CMAB solution for aDBS. Specifically, we define the context as the signals capturing irregular neuronal firing activities in the BG regions (i.e., beta-band power spectral density), while each arm signifies the (discretized) pulse frequency of the stimulation. Moreover, an {\epsilon}-exploring strategy is introduced on top of the classic Thompson sampling method, leading to an algorithm called {\epsilon}-Neural Thompson sampling ({\epsilon}-NeuralTS), such that the learned CMAB policy can better balance exploration and exploitation of the BG environment. The {\epsilon}-NeuralTS algorithm is evaluated using a computation BG model that captures the neuronal activities in PD patients' brains. The results show that our method outperforms both existing cDBS methods and CMAB baselines.

RESUMEN

Tuning the parameters of controllers to attain the best performance is a challenging task in designing effective closed-loop neuromodulation systems. In this paper, we present a distributed architecture for automated tuning and adaptation of closed-loop neuromodulation control systems. We use this approach for the automated parameter tuning of a Proportional-Integral (PI) neuromodulation controller using Bayesian optimization. We use a biophysically-grounded mean-field model of neural populations under electrical stimulation as a simulation environment for testing and prototyping the proposed framework and characterizing its performance. Our results demonstrate the feasibility of using Bayesian optimization for performance-based automated tuning of a PI controller in closed-loop set-point neuromodulation control tasks.

Asunto(s)

Aclimatación , Teorema de Bayes , Simulación por Computador , Estimulación Eléctrica

RESUMEN

A magnesium potassium phosphate hydrate-based flame-retardant coating (MKPC) is formulated by dead-burnt magnesium oxide (magnesia) and potassium dihydrogen phosphate (KH2PO4), behaving as a matrix. Constituents of the MKPC include wollastonite, vermiculite, aluminum fluoride, aluminum trihydroxide, and calcium carbonate. Some of the ingredients inter-react to produce mullite whiskers at high temperatures, despite an acid-base hydration induced reaction between magnesia and KH2PO4. The MKPC's thermal, corrosion-resistant, mechanical, and flame-resistant properties were analyzed using scanning electron microscopy, electrochemical corrosion testing, compression testing, thermogravimetric analysis, and freeze/thaw tests. The results show that with the molar ratio = 4 of magnesia to KH2PO4, MKPC demonstrates lower thermal conductivity (0.19 W/m K), along with better corrosion resistance, stronger compressive strength (10.5 MPa), and higher bonding strength (6.62 kgf/cm2) to the steel substrate. Furthermore, acceptable additives to the formulation could enhance its flame-retardancy and increase its mechanical strength as well. Mullite whisker formed from the interaction of wollastonite, aluminum trihydroxide, and aluminum fluoride acts as an outer ceramic shield that enhances mechanical strength and compactness. In addition, Mg-containing minerals with calcium carbonate treated at high temperatures, transform into magnesium calcium carbonate after releasing CO2. At the optimum composition of MKPC (magnesia/KH2PO4 molar ratio = 4; wollastonite:vermiculite = 20:10 wt.%; aluminum trihydroxide = 10 wt.%; and calcium carbonate = 5 wt.%), coated on a steel substrate, the flame-resistance limit results exhibit below 200 °C on the back surface of the steel substrate after one hour of flaming (ca. 1000 °C) on the other surface, and the flame-resistance rating results demonstrate only 420 °C on the back surface of the steel substrate after three hours of flaming (>1000 °C) on the other surface. Both requirements for the flame-resistance limit and three-hour flame-resistance rating are met with the optimum compositions, indicating that MKPC plays an effective role in establishing flame-retardancy.