RESUMEN

Low-altitude airspace is developing rapidly, but the utilization rate of airspace resources is low. Therefore, in order to solve the problem of the safe operation of the fusion of large UAVs and manned aircraft in the same airspace, this paper analyzes the theoretical calculation of the collision risk of the fusion operation of manned aircraft and UAVs at Feng Ming Airport in Zigong, verifying that while assessing the safety spacing of 10 km in the lateral direction, it further simulates the possibility of calculating the theoretical smaller safety spacing. The study will propose a new theory of error spacing safety margin and improve it according to the traditional Event collision risk model, combining the error spacing safety margin to establish an improved collision model more suitable for the fusion operation of manned and unmanned aircraft and reduce the redundancy of calculation. The error factors affecting manned and unmanned aircraft at Zigong Airport are analyzed, and theoretical calculations are analyzed by combining the actual data of Zigong Airport. Finally, the Monte Carlo simulation method is used to solve the error, substitute the calculation results, and simulate a section of the trajectory of the fusion operation for the reverse argument. The theoretical calculation results show that the collision risk from 10 km to 8 km satisfies the lateral target safety level (TSL) specified by ICAO under both traditional and improved models. The collision risk calculated by the improved model incorporating the error spacing safety margin is smaller, which enhances the safety of the model calculations. The results of the study can provide theoretical references for the fusion operation of manned and unmanned aircraft.

RESUMEN

Offline reinforcement learning (offline RL) aims to find task-solving policies from prerecorded datasets without online environment interaction. It is unfortunate that extrapolation errors can cause over-optimistic Q-value estimates when learning with a fixed dataset, limiting the performance of the learned policy. To tackle this issue, this article proposes an offline actor-critic with behavior value regularization (OAC-BVR) method. In the policy evaluation stage, the difference between the Q-function and the value of the behavior policy is considered as the regularization term, driving the learned value function to approach the value of the behavior policy. The convergence of the proposed policy evaluation with behavior value regularization (PE-BVR) and the value function difference are analyzed, respectively. Compared with existing offline actor-critic methods, the proposed OAC-BVR method integrates the value of the behavior policy, thereby simultaneously alleviating over-optimistic Q-value estimates and reducing Q-function bias. Experimental results on the D4RL MuJoCo and Maze2d datasets demonstrate the validity of the proposed PE-BVR and the performance advantage of OAC-BVR over the state-of-the-art offline RL algorithms. The code of OAC-BVR is available at https://github.com/LongyangHuang/OAC-BVR.

RESUMEN

Offline reinforcement learning (RL) aims at learning an optimal policy from a static offline data set, without interacting with the environment. However, the theoretical understanding of the existing offline RL methods needs further studies, among which the conservatism of the learned Q-function and the learned policy is a major issue. In this article, we propose a simple and efficient offline RL with relaxed conservatism (ORL-RC) framework for addressing this concern by learning a Q-function that is close to the true Q-function under the learned policy. The conservatism of learned Q-functions and policies of offline RL methods is analyzed. The analysis results support that the conservatism can lead to policy performance degradation. We establish the convergence results of the proposed ORL-RC, and the bounds of learned Q-functions with and without sampling errors, respectively, suggesting that the gap between the learned Q-function and the true Q-function can be reduced by executing the conservative policy improvement. A practical implementation of ORL-RC is presented and the experimental results on the D4RL benchmark suggest that ORL-RC exhibits superior performance and substantially outperforms existing state-of-the-art offline RL methods.

RESUMEN

With the rapid development of unmanned aerial vehicle technology and its increasing application across various fields, current airspace resources are insufficient for unmanned aerial vehicles' needs. This paper, taking Zigong General Aviation Airport in Sichuan as a case study, explores the lateral safety spacing in a mixed operation mode of unmanned aerial vehicles and manned aircraft. Currently, there are no standardized regulations for the safe spacing of the fusion operation of unmanned and manned aircraft. Theoretical research is essential to provide a reference for actual operations. It introduces the UM-Event (unmanned and manned aircraft-event) collision risk model, an adaptation of the Event collision risk model, considering factors like communication, navigation, surveillance performance, human factors, collision avoidance equipment performance, and meteorology. Safety spacing was determined via simulation experiments and actual data analysis, adhering to the target safety level (TSL). Findings indicate that surveillance performance has a minor impact on safety spacing, while communication and navigation significantly influence it. The safety spacing, influenced solely by CNS (communication performance, navigation performance, surveillance performance) and combined factors, increased from 4.42 to 4.47 nautical miles. These results offer theoretical guidance for unmanned aerial vehicle safety in non-segregated airspace.

RESUMEN

Protecting and constructing urban ecological infrastructure (UEI) is an important spatial means of improving the quality of life of urban residents, enhancing the benefits of the urban environment, and improving urban habitats. Landscape connectivity is the basis for UEI to play an ecological role. Spatial equity is an important goal for UEI to enhance urban resident happiness and sense of access and achieve environmental justice. Taking Yantai city as an example, this study analyzed the UEI morphological spatial pattern based on morphological spatial pattern analysis and calculated the spatial accessibility of UEI based on the gravity -based two-step-floating catchment area (G2SFCA) method. In addition, the study proposed the concept of accessible UEI and reflected the spatial differences in the equity of accessible UEI in urban residential areas through equity modeling that was oriented to connectivity enhancement and equity improvement. Moreover, the study integrated the location-allocation model and circuit theory to optimize the spatial layout of the existing UEI and construct an urban ecological corridor that considered landscape connectivity and leisure and recreational convenience. The results of this study showed that the proportion of bridge and island areas with connecting functions in the existing UEI in Yantai was only 10%, and they had not yet formed a complete network structure; hence, connectivity improvements were needed. In addition, the equity of accessible UEI in the residential areas of Yantai was generally good, and more than two-thirds of residents had sufficient access to UEI. However, approximately 32.7% of the residents in the four old city areas lacked a UEI distribution within their effective commuting time. Moreover, UEI layout optimization could improve the equity of accessible UEI in residential areas; however, the method of adding a new UEI was not applicable to areas with high concentrations of urban populations and stable urban spatial layouts. Furthermore, urban ecological corridors could effectively improve the connectivity and equity of UEI; however, 70% relied on existing road systems and needed to enhance their ecological attributes. The UEI equity of residential areas was highly correlated with house prices, indicating an obvious spatial injustice in the UEI layout in Yantai. Planners and governments should promote urban environmental justice through effective conservation and construction measures by incorporating established ecological and artificial infrastructures into UEI planning to achieve equitable access to UEI services for urban residents. This study provides a spatial reference and methodological support to enhance the equity and connectivity of UEI.

Asunto(s)

RESUMEN

In offline actor-critic (AC) algorithms, the distributional shift between the training data and target policy causes optimistic Q value estimates for out-of-distribution (OOD) actions. This leads to learned policies skewed toward OOD actions with falsely high Q values. The existing value-regularized offline AC algorithms address this issue by learning a conservative value function, leading to a performance drop. In this article, we propose a mild policy evaluation (MPE) by constraining the difference between the Q values of actions supported by the target policy and those of actions contained within the offline dataset. The convergence of the proposed MPE, the gap between the learned value function and the true one, and the suboptimality of the offline AC with MPE are analyzed, respectively. A mild offline AC (MOAC) algorithm is developed by integrating MPE into off-policy AC. Compared with existing offline AC algorithms, the value function gap of MOAC is bounded by the existence of sampling errors. Moreover, in the absence of sampling errors, the true state value function can be obtained. Experimental results on the D4RL benchmark dataset demonstrate the effectiveness of MPE and the performance superiority of MOAC compared to the state-of-the-art offline reinforcement learning (RL) algorithms.

RESUMEN

The accurate estimation of Q-function and the enhancement of agent's exploration ability have always been challenges of off-policy actor-critic algorithms. To address the two concerns, a novel robust actor-critic (RAC) is developed in this article. We first derive a robust policy improvement mechanism (RPIM) by using the local optimal policy about the current estimated Q-function to guide policy improvement. By constraining the relative entropy between the new policy and the previous one in policy improvement, the proposed RPIM can enhance the stability of the policy update process. The theoretical analysis shows that the incentive to increase the policy entropy is endowed when the policy is updated, which is conducive to enhancing the exploration ability of agents. Then, RAC is developed by applying the proposed RPIM to regulate the actor improvement process. The developed RAC is proven to be convergent. Finally, the proposed RAC is evaluated on some continuous-action control tasks in the MuJoCo platform and the experimental results show that RAC outperforms several state-of-the-art reinforcement learning algorithms.

RESUMEN

In this article, a novel coupled policy improvement mechanism is developed for improving policy iteration (PI) algorithms. In contrast to the common PI, the developed dual parallel policy iteration (DPPI) with coupled policy improvement mechanism consists of two parallel PIs. At each PI step, the performances of the two parallel policies are evaluated and the better one is defined as the dominant policy. Then, the dominant policy is used to guide the parallel policy improvement in a soft manner by constraining the Kullback-Liebler (KL) divergence between the dominant policy and the policy to be updated. It is proven that the convergence of DPPI can be guaranteed under the designed coupled policy improvement mechanism. Moreover, it is clearly shown that under certain conditions, the Q -functions of the two new policies obtained in each parallel policy improvement are larger than those of all the previous dominant policies, which is conductive to accelerate the PI process and improve the policy learning efficiency to some extent. Furthermore, by combining DPPI with the twin delay deep deterministic (TD3) policy gradient, we propose a reinforcement learning (RL) algorithm: parallel TD3 (PTD3). Experimental results on continuous-action control tasks in the MuJoCo and OpenAI Gym platforms show that the proposed PTD3 outperforms the state-of-the-art RL algorithms.

RESUMEN

Exploring the changes in ecological networks (ENs), its eco-environment effects and the differences in urban agglomerations in various urbanization stages are important for achieving sustainable ecosystem management and a better layout of ecological network. In this study, China's three typical urbanization agglomerations, Beijing-Tianjin-Hebei urban agglomeration (BTH), Yangtze River Delta urban agglomeration (YRD), and Pearl River Delta urban agglomeration (PRD), were selected as the study area. Spatiotemporal changes in ENs, the changing patterns, its eco-environment effects, and impacts of rapid urbanization were analyzed by environment indices, buffer analysis, and correlation analysis. The results showed a great lost in ENs from 2000 to 2015. Four patterns were seen in changing ENs: decomposition process (DP), internal change process (ICP), polycondensation process (PP), and external change process (ECP). ICP was dominated in YRD and PRD. ECP was the main pattern in core areas of BTH. The correlation analysis with YRD as the example showed that the changes in ENs had a certain impact on the eco-environment, especially in the 10-km buffer zone. The decrease of ENs was related to the increase of developed land, and the closer to the core area, the higher the correlation coefficient was. Reduction of ENs would slow down to a certain extent, when the agglomeration is in a higher urbanization stage. Different directions of restoration and optimization of ENs were proposed for the three urban agglomerations. The study will provide support for sustainable management and restoration and optimization of ENs for China's agglomerations.

Asunto(s)

RESUMEN

Ecological infrastructure (EI) planning can promote regional nature conservation efficiency and enhance ecosystem functions. Watershed-scale EI research is a research hotspot in landscape ecology. This study proposed a method framework to develop EI planning in large river basins based on the connectivity of ecological processes and the integrity of ecosystems in the whole basin, as well as the typical ecological problems in each sub-basin. The framework included three parts: determining the protective EI i.e. the spatial range of the watershed ecological networks; quantifying and mapping the functional EI i.e. the typical ecosystem functions within each sub-basin; and integrating ecological networks and key ecosystem function area into an EI planning based on their spatial overlap and functional synergy. The method framework was applied in the Yellow River Basin. Results showed that spatial range of ecological networks of the basin accounted for 35.8% of the study area. Key ecosystem function area including ecosystem function important area and ecosystem function improvement area accounted for 35.6%. Spatial overlay analysis of ecological networks and key ecosystem function areas showed that they overlap spatially and have synergistic effects functionally, but core habitats existed less human activities compared to key ecosystem function area. By integrating ecological networks and key ecosystem function areas, EI planning including four spatial types: water system, core habitat area, important area for ecological function maintenance, priority area for ecological function improvement. The corresponding protection and development measures were formulated. In addition, along with the improvement of ecosystem function in the basin, sediment content, sand transport and other indicators in Yellow River has been obviously improved. It indicated that the EI construction in the basin had positive significance for river governance. Results showed that the EI planning method could improve both the spatial accuracy of nature conservation in the watershed and promote the specific ecosystem functions. It is also applicable to river management and watershed territorial spatial planning in other large river basins.

Asunto(s)

RESUMEN

In recent years, the proximal policy optimization (PPO) algorithm has received considerable attention because of its excellent performance in many challenging tasks. However, there is still a large space for theoretical explanation of the mechanism of PPO's horizontal clipping operation, which is a key means to improve the performance of PPO. In addition, while PPO is inspired by the learning theory of trust region policy optimization (TRPO), the theoretical connection between PPO's clipping operation and TRPO's trust region constraint has not been well studied. In this article, we first analyze the effect of PPO's clipping operation on the objective function of conservative policy iteration, and strictly give the theoretical relationship between PPO and TRPO. Then, a novel first-order policy gradient algorithm called authentic boundary PPO (ABPPO) is proposed, which is based on the authentic boundary setting rule. To ensure the difference between the new and old policies is better kept within the clipping range, by borrowing the idea of ABPPO, we proposed two novel improved PPO algorithms called rollback mechanism-based ABPPO (RMABPPO) and penalized point policy difference-based ABPPO (P3DABPPO), which are based on the ideas of rollback clipping and penalized point policy difference, respectively. Experiments on the continuous robotic control tasks implemented in MuJoCo show that our proposed improved PPO algorithms can effectively improve the learning stability and accelerate the learning speed compared with the original PPO.

Asunto(s)

RESUMEN

Rapid urbanization and industrialization in the coastal zone have caused increasingly serious impacts on coastal ecosystems. It is necessary to assess the ecological risk caused by human activities to determine key areas of terrestrial-oceanic ecosystems preservation and restoration to ensure sustainable ecological management in the coastal zone. Key areas of ecosystem preservation and restoration were studied through the assessment of the impacts of ecological pressure sources related to human activities from the perspective of terrestrial-oceanic ecosystems, using the habitat risk assessment (HRA) and habitat quality (HQ) models in the Chinese coastal zone. The results showed that the impact of human activities on the terrestrial ecosystems in the South of China was significantly lower than that in the North. An improvement rate of habitat quality was noticed only in the south and central coastal areas when further away from industrial land. Agricultural production, urban expansion, and industrial pollution had major negative impacts on the habitat quality of terrestrial ecosystems in the Chinese coastal zone, and also threatened the health of marine ecosystems. The ecological risks caused by human activities in the offshore areas of northern Shandong and eastern Jiangsu were relatively low. Mineral development in the north, excessive nitrogen and phosphorus emissions from agricultural production in the south, and port operations were important drivers of increased ecological risks in offshore areas. There were regional spatial differences in the key ecosystem preservation and restoration areas. The provinces of Shandong, Jiangsu, Hebei, Liaoning, and Guangdong are key areas for strengthening the preservation and restoration of terrestrial-oceanic ecosystems. This study provides a reference for large-scale territorial spatial planning and ecosystems conservation.

RESUMEN

The ecological security pattern aims to maintain the integrity of landscape pattern and the continuity of ecological process to ensure urban ecological security. Current ecological source identification methods lack the consideration for ecological land degradation risks and human ecological needs. Taking Wuhan City as an example, the comprehensive value of ecological land was calcula-ted from three aspects, including ecological land quality, ecological degradation risk, and ecological demand to identify ecological source areas. Basic ecological resistance surfaces were constructed based on land use types and nighttime light data. Potential ecological corridors were identified based on the minimum cumulative resistance model. Ecological "pinch points" were recognized based on circuit theory. After that, we built a "four horizontal, three vertical and ten groups" ecological safety pattern in Wuhan. The results showed that the area of ecological land in Wuhan was 2138.2 km2, accounting for 24.9% of the total area of the city. These areas were primarily attributed to waters and foress, clustering in the north and south of the city. The total length of ecological corridors was 1222.42 km, with 566.75 km aquatic corridors and 655.67 km land corridors. The aquatic corridors traversed the city area to form a double cross-shaped pattern. The land corridors circularly distributed around the city. Comprehensively, the "four horizontal and three vertical" spatial pattern had appeared. There were a total of 44 ecological pinch points on the ecological corridor, distributed circularly with the central city as the kernel. The extant protection space had almost been included in the identified ecological source area, supporting the ecological significance of the identification framework. It would help to provide a quantitative framework for the construction of ecological secu-rity pattern in the metropolitan area and to guide the relevant urban spatial planning.

Asunto(s)