RESUMEN
Although deep deterministic policy gradient (DDPG) algorithm gets widespread attention as a result of its powerful functionality and applicability for large-scale continuous control, it cannot be denied that DDPG has problems such as low sample utilization efficiency and insufficient exploration. Therefore, an improved DDPG is presented to overcome these challenges in this article. Firstly, an optimizer based on fractional gradient is introduced into the algorithm network, which is conductive to increase the speed and accuracy of training convergence. On this basis, high-value experience replay based on weight-changed priority is proposed to improve sample utilization efficiency, and aiming to have a stronger exploration of the environment, an optimized exploration strategy for boundary action space is adopted. Finally, our proposed method is tested through the experiments of gym and pybullet platform. According to the results, our method speeds up the learning process, obtains higher average rewards in comparison with other algorithms.
RESUMEN
In this paper, the composite robust control problem of uncertain nonlinear systems with unmatched disturbances is investigated. In order to improve the robust control performance, the integral sliding mode control method is considered together with H∞ control for nonlinear systems. By designing a disturbance observer with a new structure, the estimations of disturbances can be obtained with small errors, which are used to construct sliding mode control policy and avoid high gains. On the basis of ensuring the accessibility of specified sliding surface, the guaranteed cost control problem of nonlinear sliding mode dynamics is considered. To overcome the difficulty of robust control design caused by nonlinear characteristics, a modified policy iteration method based on sum of squares is proposed to solve the H∞ control policy of the nonlinear sliding mode dynamics. Finally, the effectiveness of the proposed robust control method is verified by simulation tests.
RESUMEN
This article concentrates on the event-based collaborative design for strict-feedback systems with uncertain nonlinearities. The controller is designed based on neural network (NN) weights adaptive law. The controller and NN weights adaptive law are only updated at the triggering instants determined by a novel composite triggering threshold. Considering the conservativeness of event condition, the state-model error is integrated into constructing the composite condition and NN weights adaptive law. In the context of the proposed mechanism, the requirements of system information and the allowable range of event-triggering error are relaxed. The number of triggering instants is greatly reduced without deteriorating the system performance. Moreover, the stability of the closed-loop is proved by the Lyapunov method following time-interval and sampling instants. Simulation results show the effectiveness of the scheme proposed in this article.
RESUMEN
In this article, the problem of H∞ codesign for nonlinear control systems with unmatched uncertainties and adjustable parameters is investigated. The main purpose is to solve the adjustable parameters and H∞ controller simultaneously so that better robust control performance can be achieved. By introducing a bounded function and defining a special cost function, the problem of solving the Hamilton-Jacobi-Isaacs equation is transformed into an optimization problem with nonlinear inequality constraints. Based on the sum of squares technique, a novel policy iteration algorithm is proposed to solve the problem of the H∞ codesign. Moreover, one modified algorithm for optimizing the robust performance index is given. The convergence and the performance improvement of new iteration policy algorithms are proved. Simulation results are presented to demonstrate the effectiveness of the proposed algorithms.
RESUMEN
In this paper, the problem of adaptive actor-critic (AC) tracking control is investigated for a class of continuous-time nonlinear systems with unknown nonlinearities and quantized inputs. Different from the existing results based on reinforcement learning, the tracking error constraints are considered and new critic functions are constructed to improve the performance further. To ensure that the tracking errors keep within the predefined time-varying boundaries, a tracking error transformation technique is used to constitute an augmented error system. Specific critic functions, rather than the long-term cost function, are introduced to supervise the tracking performance and tune the weights of the AC neural networks (NNs). A novel adaptive controller with a special structure is designed to reduce the effect of the NN reconstruction errors, input quantization, and disturbances. Based on the Lyapunov stability theory, the boundedness of the closed-loop signals and the desired tracking performance can be guaranteed. Finally, simulations on two connected inverted pendulums are given to illustrate the effectiveness of the proposed method.
RESUMEN
This paper is concerned with the problem of active complementary control design for affine nonlinear control systems with actuator faults. The outage and loss of effectiveness fault cases are considered. In order to achieve the performance enhancement of the faulty control system, the complementary control scheme is designed in two steps. Firstly, a novel fault estimation scheme is developed. Then, by using the fault estimations to reconstruct the faulty system dynamics and introducing a cost function as the optimization objective, a nearly optimal complementary control is obtained online based on the adaptive dynamic programming (ADP) method. Unlike most of the previous ADP methods with the addition of a probing signal, new adaptive weight update laws are derived to guarantee the convergence of neural network weights and the stability of the closed-loop system, which strongly supports the online implementation of the ADP method. Finally, two simulation examples are given to illustrate the performance and effectiveness of the proposed method.
RESUMEN
The state inequality constraints have been hardly considered in the literature on solving the nonlinear optimal control problem based the adaptive dynamic programming (ADP) method. In this paper, an actor-critic (AC) algorithm is developed to solve the optimal control problem with a discounted cost function for a class of state-constrained nonaffine nonlinear systems. To overcome the difficulties resulting from the inequality constraints and the nonaffine nonlinearities of the controlled systems, a novel transformation technique with redesigned slack functions and a pre-compensator method are introduced to convert the constrained optimal control problem into an unconstrained one for affine nonlinear systems. Then, based on the policy iteration (PI) algorithm, an online AC scheme is proposed to learn the nearly optimal control policy for the obtained affine nonlinear dynamics. Using the information of the nonlinear model, novel adaptive update laws are designed to guarantee the convergence of the neural network (NN) weights and the stability of the affine nonlinear dynamics without the requirement for the probing signal. Finally, the effectiveness of the proposed method is validated by simulation studies.
RESUMEN
This paper is concerned with the problem of integral sliding-mode control for a class of nonlinear systems with input disturbances and unknown nonlinear terms through the adaptive actor-critic (AC) control method. The main objective is to design a sliding-mode control methodology based on the adaptive dynamic programming (ADP) method, so that the closed-loop system with time-varying disturbances is stable and the nearly optimal performance of the sliding-mode dynamics can be guaranteed. In the first step, a neural network (NN)-based observer and a disturbance observer are designed to approximate the unknown nonlinear terms and estimate the input disturbances, respectively. Based on the NN approximations and disturbance estimations, the discontinuous part of the sliding-mode control is constructed to eliminate the effect of the disturbances and attain the expected equivalent sliding-mode dynamics. Then, the ADP method with AC structure is presented to learn the optimal control for the sliding-mode dynamics online. Reconstructed tuning laws are developed to guarantee the stability of the sliding-mode dynamics and the convergence of the weights of critic and actor NNs. Finally, the simulation results are presented to illustrate the effectiveness of the proposed method.