RESUMO

This study evaluated the hardness of a composite resin used for root reinforcement, considering the light-curing time, root canal region and ageing due to long-term storage. Twenty incisor roots were reinforced using composite resin, varying the photopolymerisation time (40 or 120 s). Following fibre post cementation, the roots were transversely sectioned into coronal, middle and apical regions. Composite hardness was measured initially and after 18 months of water storage. Data underwent repeated measures analysis of variance and Tukey's post hoc tests. The factors 'light-curing time', 'root region' and 'ageing' affected the hardness. Significant interactions were observed between 'light-curing time × root region' and 'ageing × light-curing time'. Regardless of time, resin hardness in the apical region was lower. After ageing, hardness in the coronal and middle regions decreased when the light-curing time was 40 s, while no significant effect on hardness was noted with a light-curing time of 120 s.

RESUMO

Reinforcement learning (RL) methods have helped to define the state of the art in the field of modern artificial intelligence, mostly after the breakthrough involving AlphaGo and the discovery of novel algorithms. In this work, we present a RL method, based on Q-learning, for the structural determination of adsorbate@substrate models in silico, where the minimization of the energy landscape resulting from adsorbate interactions with a substrate is made by actions on states (translations and rotations) chosen from an agent's policy. The proposed RL method is implemented in an early version of the reinforcement learning software for materials design and discovery (RLMaterial), developed in Python3.x. RLMaterial interfaces with deMon2k, DFTB+, ORCA, and Quantum Espresso codes to compute the adsorbate@substrate energies. The RL method was applied for the structural determination of (i) the amino acid glycine and (ii) 2-amino-acetaldehyde, both interacting with a boron nitride (BN) monolayer, (iii) host-guest interactions between phenylboronic acid and ß-cyclodextrin and (iv) ammonia on naphthalene. Density functional tight binding calculations were used to build the complex search surfaces with a reasonably low computational cost for systems (i)-(iii) and DFT for system (iv). Artificial neural network and gradient boosting regression techniques were employed to approximate the Q-matrix or Q-table for better decision making (policy) on next actions. Finally, we have developed a transfer-learning protocol within the RL framework that allows learning from one chemical system and transferring the experience to another, as well as from different DFT or DFTB levels.

RESUMO

In this study, we introduce an innovative policy in the field of reinforcement learning, specifically designed as an action selection mechanism, and applied herein as a selector for binarization schemes. These schemes enable continuous metaheuristics to be applied to binary problems, thereby paving new paths in combinatorial optimization. To evaluate its efficacy, we implemented this policy within our BSS framework, which integrates a variety of reinforcement learning and metaheuristic techniques. Upon resolving 45 instances of the Set Covering Problem, our results demonstrate that reinforcement learning can play a crucial role in enhancing the binarization techniques employed. This policy not only significantly outperformed traditional methods in terms of precision and efficiency, but also proved to be extensible and adaptable to other techniques and similar problems. The approach proposed in this article is capable of significantly surpassing traditional methods in precision and efficiency, which could have important implications for a wide range of real-world applications. This study underscores the philosophy behind our approach: utilizing reinforcement learning not as an end in itself, but as a powerful tool for solving binary combinatorial problems, emphasizing its practical applicability and potential to transform the way we address complex challenges across various fields.

RESUMO

Behavioral momentum theory (BMT) provides a theoretical and methodological framework for understanding how differentially maintained operant responding resists disruption. A common way to test operant resistance involves contingencies with suppressive effects, such as extinction or prefeeding. Other contingencies with known suppressive effects, such as response-cost procedures arranged as point-loss or increases in response force, remain untested as disruptive events within the BMT framework. In the present set of three experiments, responding of humans was maintained by point accumulation programmed according to a multiple variable-interval (VI) VI schedule with different reinforcement rates in either of two components. Subsequently, subtracting a point following each response (Experiment 1) or increasing the force required for the response to be registered (Experiments 2 and 3 decreased response rates, but responding was less disrupted in the component associated with the higher reinforcement rate. The point-loss contingency and increased response force similarly affected response rates by suppressing responding and human persistence, replicating previous findings with humans and nonhuman animals when other types of disruptive events (e.g., extinction and prefeeding) were investigated. The present findings moreover extend the generality of the effects of reinforcement rate on persistence, and thus BMT, extending the analysis of resistance to two well-known manipulations used to reduce responding in the experimental analysis of behavior.

Assuntos

RESUMO

BACKGROUND: The literature is scanty regarding the effect of radiation therapy (RT) on the mechanical properties of immature permanent teeth. AIM: To evaluate the effect of RT on the fracture resistance of simulated immature teeth submitted to different types of root reinforcement. DESIGN: Sixty-four human teeth simulating the Cvek stage 3 of root development were distributed into eight groups (n = 8), according to exposure or not to RT (70 Gy) and the root reinforcement method: Group NR (control)-no reinforcement/no RT; Group NR + RT (control)-no reinforcement/RT; Group PO-tricalcium silicate-based cement (TS) apical plug/canal obturation/no RT; Group PO + RT-TS apical plug/canal obturation/RT; Group TS-canal filling with TS/no RT; Group TS + RT-canal filling with TS/RT; Group FP-TS apical plug/fibreglass post/no RT; and Group FP + RT-TS apical plug/fibreglass post/RT. Fracture resistance was determined using a universal testing machine (0.5 mm/min). RESULTS: In the intergroup comparison, nonirradiated teeth had higher fracture resistance (p < .05). Groups FP and FP + RT had higher fracture resistance (p < .001). CONCLUSION: Radiotherapy affected the fracture resistance of simulated immature teeth. Reinforcement with fibreglass posts increased the fracture resistance, regardless of the radiation.

Assuntos

RESUMO

The conducted investigation encompassed the comprehensive integration of mechanical, environmental, chemical, and microstructural evaluations of a composite amalgamating sandy soil and a synthetic polymer at two distinct concentrations (2.5% and 5%) across multiple curing intervals (0, 1, 2, 4, 7, 15, 30, and 45 days). The studied soil originates from an environmentally significant protected area in Brazil. The implementation of mechanisms for soil improvement in the region must adhere to technical criteria without causing environmental harm. Direct shear testing was conducted, permeability was assessed, and microstructure analysis and XRD and XRF/EDX studies of both the soil and composites were conducted. It was observed that longer curing times yielded improved results in shear stress, friction angle, and cohesive intercept, with SP_5% exhibiting the highest values compared with the soil and SP_2.5%. Adding the polymeric solution to the soil contributed to cementation and cohesion gains in the substrate. Through microstructural characterization, the polymer's role as a cementing agent for the grains is evident, forming a film on the grains and binding them together. Based on the analyses and studies conducted in the research, it can be concluded that there is technical feasibility for applying the polymeric solution at both dosages in geotechnical projects.

RESUMO

Introduction: This study analysed the psychometric properties of the Reward Probability Index (RPI) in an online Colombian sample with 1129 participants. Method: To conduct a cross-validation study, the sample was randomly divided into two subsamples. An exploratory factor analysis was conducted with the first subsample yielding a two-factor structure. Then, the fit of this two-factor model was tested on the second subsample by conducting a confirmatory factor analysis. Results: This model obtained a good fit to the data and measurement invariance across gender was observed. The RPI also showed good internal consistency according to both Cronbach's alpha and McDonald's omega, scoring .88 in both cases. The RPI demonstrated convergent construct validity given its correlations with other related measures such as the Environmental Reward Observation Scale (r = .81), and the full version of the Behavioral Activation Scale for Depression (r = .71). Conclusions: The RPI showed good psychometric properties in this Colombian sample.

Introducción: Este artículo tuvo como objetivo analizar las propiedades psicométricas del Índice de Probabilidad de Recompensa (RPI) en una muestra colombiana en línea con 1129 participantes. Método: Para realizar un estudio de validación cruzada, la muestra se dividió aleatoriamente en dos submuestras. Se realizó un análisis factorial exploratorio con la primera submuestra que arrojó una estructura de dos factores. Luego, se probó el ajuste de este modelo de dos factores en la segunda submuestra mediante la realización de un análisis factorial confirmatorio. Resultados: Este modelo obtuvo un buen ajuste a los datos y se observó invarianza de medida entre sexos. El RPI también mostró buena consistencia interna según el alfa de Cronbach y el omega de McDonald (.88 en ambos casos) y validez de constructo convergente dadas las correlaciones con otras medidas relacionadas como la Escala de Observación de Recompensa Ambiental (r = .81), y la versión de la Escala de Activación Conductual para la Depresión (r = .71). Conclusiones: el RPI mostró buenas propiedades psicométricas en esta muestra colombiana.

Assuntos

RESUMO

There is only a very short reaction time for people to find the best way out of a building in a fire outbreak. Software applications can be used to assist the rapid evacuation of people from the building; however, this is an arduous task, which requires an understanding of advanced technologies. Since well-known pathway algorithms (such as, Dijkstra, Bellman-Ford, and A*) can lead to serious performance problems, when it comes to multi-objective problems, we decided to make use of deep reinforcement learning techniques. A wide range of strategies including a random initialization of replay buffer and transfer learning were assessed in three projects involving schools of different sizes. The results showed the proposal was viable and that in most cases the performance of transfer learning was superior, enabling the learning agent to be trained in times shorter than 1 min, with 100% accuracy in the routes. In addition, the study raised challenges that had to be faced in the future.

Assuntos

RESUMO

This article delves into the complex world of quantum games in multi-agent settings, proposing a model wherein agents utilize gradient-based strategies to optimize local rewards. A learning model is introduced to focus on the learning efficacy of agents in various games and the impact of quantum circuit noise on the performance of the algorithm. The research uncovers a non-trivial relationship between quantum circuit noise and algorithm performance. While generally an increase in quantum noise leads to performance decline, we show that low noise can unexpectedly enhance performance in games with large numbers of agents under some specific circumstances. This insight not only bears theoretical interest, but also might have practical implications given the inherent limitations of contemporary noisy intermediate-scale quantum (NISQ) computers. The results presented in this paper offer new perspectives on quantum games and enrich our understanding of the interplay between multi-agent learning and quantum computation. Both challenges and opportunities are highlighted, suggesting promising directions for future research in the intersection of quantum computing, game theory and reinforcement learning.

RESUMO

Automated industrial processes require a controller to obtain an output signal similar to the reference indicated by the user. There are controllers such as PIDs, which are efficient if the system does not change its initial conditions. However, if this is not the case, the controller must be retuned, affecting production times. In this work, an adaptive PID controller is developed for a DC motor speed plant using an artificial intelligence algorithm based on reinforcement learning. This algorithm uses an actor-critic agent, where its objective is to optimize the actor's policy and train a critic for rewards. This will generate the appropriate gains without the need to know the system. The Deep Deterministic Policy Gradient with Twin Delayed (DDPG TD3) was used, with a network composed of 300 neurons for the agent's learning. Finally, the performance of the obtained controller is compared with a classical control one using a cost function.

RESUMO

Recent studies and literature reviews have shown promising results for 3GPP system solutions in unlicensed bands when coexisting with Wi-Fi, either by using the duty cycle (DC) approach or licensed-assisted access (LAA). However, it is widely known that general performance in these coexistence scenarios is dependent on traffic and how the duty cycle is adjusted. Most DC solutions configure their parameters statically, which can result in performance losses when the scenario experiences changes on the offered data. In our previous works, we demonstrated that reinforcement learning (RL) techniques can be used to adjust DC parameters. We showed that a Q-learning (QL) solution that adapts the LTE DC ratio to the transmitted data rate can maximize the Wi-Fi/LTE-Unlicensed (LTE-U) aggregated throughput. In this paper, we extend our previous solution by implementing a simpler and more efficient algorithm based on multiarmed bandit (MAB) theory. We evaluate its performance and compare it with the previous one in different traffic scenarios. The results demonstrate that our new solution offers improved balance in throughput, providing similar results for LTE and Wi-Fi, while still showing a substantial system gain. Moreover, in one of the scenarios, our solution outperforms the previous approach by 6% in system throughput. In terms of user throughput, it achieves more than 100% gain for the users at the 10th percentile of performance, while the old solution only achieves a 10% gain.

RESUMO

Several closed or hybrid loop controllers for Blood Glucose (BG) regulation, which are also known as Artificial Pancreas (AP) Systems or Automated Insulin Delivery systems (AIDs), are in development worldwide. Most AIDs are designed and evaluated for short-term performance, with a particular emphasis on the post-meal period. However, if controllers are not adapted properly to account for variations in physiology that affect Insulin Sensitivity (IS), the AIDs may perform inadequately. In this work, the performance of two Reinforcement Learning (RL) agents trained under both piecewise and continuous reward functions is evaluated in-silico for long-term adaptation of a Fully Automated Insulin Delivery (fAID) system. An automatic adaptive discretization scheme that expands the state space as needed is also implemented to avoid disproportionate state space exploration. The proposed agents are evaluated for long-term adaptation of the Automatic Regulation of Glucose (ARG) algorithm, considering variations in IS. Results show that both RL agents have improved performance compared to a rule-based decision-making approach and the baseline controller for the majority of the adult population. Moreover, the use of a continuous shaped reward function proves to enhance the performance of the agents further than a piecewise one.

RESUMO

Natural fibers have some advantages in comparison to synthetic fibers, especially because they are more environmentally friendly. For this reason, using them as a reinforcement for polymeric matrices is growing exponentially. However, they present the disadvantage of having the hydrophilic nature, which strongly reduces the interface interaction. Sedge fibers have been investigated when reinforcing an epoxy matrix in terms of ballistic properties and mechanical performance. Aiming to enhance the fiber-matrix interface, an alkali treatment was proposed. The group conditions were divided into three NaOH concentrations (3%, 5%, and 10%), as well as the three periods of immersion (24, 48, and 72 h). Therefore, nine different conditions were investigated in terms of their thermal behaviors, chemical structures, physical structures, and morphological aspects. Based on TGA curves, it could be noticed that treatments related to 3% NaOH for 24 h and 48 h exhibited better thermal stability properties. For the time of 48 h, better thermal stability with for a decay of the thermal DSC curve was shown for all treatment conditions. The FTIR spectra has shown a reduction of waxes for higher NaOH concentrations. The XRD diffractogram exhibited an increase in the crystallinity index only for 5% NaOH and an immersion time of 48 h. The morphological aspects of fibers treated with 5% and 10% of NaOH have shown that the treatments have damaged the fiber, which highlighted the crystallinity index reductions.

RESUMO

In the basic verbal task from Piaget, when a relation of the form if A > B and B > C is given, a logical inference A > C is expected. This process is called transitive inference (TI). The adapted version for animals involves the presentation of a simultaneous discrimination between stimuli pairs. In this way, when A+B-, B+C-, C+D-, D+E- is trained, a B>D preference is expected, assuming that if A>B>C>D>E, then B>D. This effect has been widely reported using several procedures and different species. In the current experiment TI was evaluated employing probabilistic reinforcement. Thus, for the positive stimuli a .7 probability was administered and for the negative stimuli a .3 probability was administered. Under this arrangement the relation A>B>C>D>E is still allowed, but TI becomes more difficult. Five pigeons (Columba Livia) were exposed to the mentioned arrangement. Only one pigeon reached the criterion in C+D- discrimination, whereas the remaining did not. Only the one who successfully solved C+D- was capable of learning TI, whereas the others were not. Additionally, it was found that correct response ratios did not predict BD performance. Consequently, probabilistic reinforcement disrupted TI, but some positional ordering was retained in the test. The results suggest that TI might be affected by associative strength but also by the positional ordering of the stimuli. The discussion addresses the two main accounts of TI: the associative account and the ordinal representation account.

RESUMO

In recent years, hand gesture recognition (HGR) technologies that use electromyography (EMG) signals have been of considerable interest in developing human-machine interfaces. Most state-of-the-art HGR approaches are based mainly on supervised machine learning (ML). However, the use of reinforcement learning (RL) techniques to classify EMGs is still a new and open research topic. Methods based on RL have some advantages such as promising classification performance and online learning from the user's experience. In this work, we propose a user-specific HGR system based on an RL-based agent that learns to characterize EMG signals from five different hand gestures using Deep Q-network (DQN) and Double-Deep Q-Network (Double-DQN) algorithms. Both methods use a feed-forward artificial neural network (ANN) for the representation of the agent policy. We also performed additional tests by adding a long-short-term memory (LSTM) layer to the ANN to analyze and compare its performance. We performed experiments using training, validation, and test sets from our public dataset, EMG-EPN-612. The final accuracy results demonstrate that the best model was DQN without LSTM, obtaining classification and recognition accuracies of up to 90.37%±10.7% and 82.52%±10.9%, respectively. The results obtained in this work demonstrate that RL methods such as DQN and Double-DQN can obtain promising results for classification and recognition problems based on EMG signals.

Assuntos

RESUMO

Although the consumption of carbohydrates is needed for survival, their potent reinforcing properties drive obesity worldwide. In turn, sugar overconsumption reveals a major role for brain reward systems in regulating sugar intake. However, it remains elusive how different cell types within the reward circuitries control the initiation and termination of sugary meals. Here, we identified the distinct nucleus accumbens cell types that mediate the chemosensory versus postprandial properties of sweet sugars. Specifically, D1 neurons enhance sugar intake via specialized connections to taste ganglia, whereas D2 neurons mediate the termination of sugary meals via anatomical connections to circuits involved in appetite suppression. Consistently, D2, but not D1, neurons partially mediate the satiating effects of glucagon-like peptide 1 (GLP-1) agonists. Thus, these nucleus accumbens cell types function as a behavioral switch, enabling positive versus negative control over sugar intake. Our study contributes to unveiling the cellular and circuit substrates of sugar overconsumption.

Assuntos

RESUMO

We present the mathematical description of feedback functions of variable interval and variable differential reinforcement of low rates as functions of schedule size only. These results were obtained using an R script named Beak, which was built to simulate rates of behavior interacting with simple schedules of reinforcement. Using Beak, we have simulated data that allow an assessment of different reinforcement feedback functions. This was made with unparalleled precision, as simulations provide huge samples of data and, more importantly, simulated behavior is not changed by the reinforcement it produces. Therefore, we can vary response rates systematically. We've compared different reinforcement feedback functions for random interval schedules, using the following criteria: meaning, precision, parsimony, and generality. Our results indicate that the best feedback function for the random interval schedule was published by Baum (1981). We also propose that the model used by Killeen (1975) is a viable feedback function for the random differential reinforcement of low rates schedule. We argue that Beak paves the way for greater understanding of schedules of reinforcement, addressing still open questions about quantitative features of simple schedules. Also, Beak could guide future experiments that use schedules as theoretical and methodological tools.

Assuntos

RESUMO

Objective: This study provides a first approach to the use of the Multiple-Choice Procedure in social media networks use, as well as empirical evidence for the application of the Behavioral Perspective Model to digital consumption behavior in young users in conjunction with a methodology based on behavioral economics. Participants/methods: The participants were part of a large university in Bogotá, Colombia, and they received an academic credit once they completed the online questionnaire. A total of 311 participants completed the experiment. Of the participants, 49% were men with a mean age of 20.6 years (SD = 3.10, Range = 15-30); 51% were women with a mean age of 20.2 years (SD = 2.84, Range = 15-29). Results: Among the total participants, 40% reported that they used social networks between 1 and 2 h a day, 38% between 2 and 3 h, 16% for 4 h or more, and the remaining 9% used them for 1 h or less per day. The factorial analysis of variance (ANOVA) allowed us to identify a statistically significant effect of the delay of the alternative reinforcer, that is, the average crossover points were higher when the monetary reinforcer was delayed 1 week, compared to the immediate delivery of the monetary reinforcer. There was no statistically significant effect of the interaction between the magnitude of the reinforcer and the delay time of the alternative reinforcer. Conclusions: This study supports the relative reinforcing value of an informational reinforcement consequence such as social media use, which is sensitive to both the magnitude of reinforcement and the delay in delivery as individual factors. The findings on reinforcer magnitude and delay effects are consistent with previous research that have applied behavioral economics to the study of non-substance-related addictions.

Assuntos

RESUMO

One of the current challenges is to add value to agro-industrial wastes, and the cocoa industry generates about 10 tons of cocoa pod husks in Colombia for each ton of cocoa beans, which are incinerated and cause environmental damage. This study characterized the Colombian cocoa pod husk (CPH) and to isolate and characterize cellulose microfibers (tCPH) extracted via chemical treatment and pressure. Chemical and physical analyses of CPH were performed, and a pretreatment method for CPH fibers was developed, which is followed by a hydrolysis method involving high pressure in an autoclave machine with an alkaline medium (6% NaOH), and finally, bleaching of the fiber to obtain tCPH. The tCPH cellulose microfibers were also chemically and physically analyzed and characterized by infrared spectroscopy (FTIR), X-ray diffraction (XRD), scanning electron microscopy (SEM), and thermo-gravimetric analysis (TGA). Chemical and physical characterization showed a decrease in lignin content in tCPH. FTIR analysis showed the absence of some peaks in tCPH with respect to the CPH spectrum; XRD results showed an increase in crystallinity for tCPH compared to CPH, due to a higher presence of crystalline cellulose in tCPH. SEM images included a control fiber treated without high pressure (tCPHnpe), and agglomerated fibers were observed, whereas cellulose microfibers with a mean diameter of 10 ± 2.742 µm were observed in tCPH. Finally, with TGA and DTGA it was confirmed that in tCPH, the hemicellulose and lignin were removed more successfully than in the control fiber (tCPHnpe), showing that the treatment with pressure was effective at isolating the cellulose microfibers from cocoa pod husk.

RESUMO

In recent years, cybersecurity has been strengthened through the adoption of processes, mechanisms and rapid sources of indicators of compromise in critical areas. Among the most latent challenges are the detection, classification and eradication of malware and Denial of Service Cyber-Attacks (DoS). The literature has presented different ways to obtain and evaluate malware- and DoS-cyber-attack-related instances, either from a technical point of view or by offering ready-to-use datasets. However, acquiring fresh, up-to-date samples requires an arduous process of exploration, sandbox configuration and mass storage, which may ultimately result in an unbalanced or under-represented set. Synthetic sample generation has shown that the cost associated with setting up controlled environments and time spent on sample evaluation can be reduced. Nevertheless, the process is performed when the observations already belong to a characterized set, totally detached from a real environment. In order to solve the aforementioned, this work proposes a methodology for the generation of synthetic samples of malicious Portable Executable binaries and DoS cyber-attacks. The task is performed via a Reinforcement Learning engine, which learns from a baseline of different malware families and DoS cyber-attack network properties, resulting in new, mutated and highly functional samples. Experimental results demonstrate the high adaptability of the outputs as new input datasets for different Machine Learning algorithms.