RESUMEN

Fusing three-dimensional (3D) and multispectral (MS) imaging data holds promise for high-throughput, comprehensive plant phenotyping to decipher genome-to-phenome knowledge. The acquisition of high-quality 3D multispectral point clouds (3DMPCs) of plants remains challenging because of poor 3D data quality and limited radiometric calibration methods for plants with complex canopy structure. We proposed a novel 3D spatial-spectral data fusion approach to collect high-quality 3DMPCs of plants by integrating the next-best-view (NBV) planning for adaptive data acquisition and Neural REference Field (NeREF) for radiometric calibration. Our approach was used to acquire 3DMPCs of perilla, tomato and rapeseed plants with diverse plant architecture and leaf morphological features and evaluated by the accuracy of chlorophyll content and equivalent water thickness (EWT) estimation. Results showed that the completeness of plant point clouds collected by our approach was improved by an average of 23.6% compared with the fixed viewpoints alone. The NeREF-based radiometric calibration with the hemispherical reference outperformed the conventional calibration method by reducing the root mean square error (RMSE) of 58.93% for extracted reflectance spectra. The RMSE for chlorophyll content and EWT predictions decreased by 21.25% and 14.13% using the partial least squares regression (PLSR) with the generated 3DMPCs. Our study provided an effective and efficient way to collect high-quality 3DMPCs of plants under the natural light condition, which improves the accuracy and comprehensiveness of phenotyping plant morphological and physiological traits and facilitates plant biology and genetic studies and breeding programs.

RESUMEN

The common non-contact, automatic body size measurement methods based on the whole livestock point cloud are complex and prone to errors. Therefore, a cattle body measuring system is proposed. The system includes a new algorithm called dynamic unbalanced octree grouping (DUOS), based on PointNet++, and an efficient method of body size measurement based on segmentation results. This system is suitable for livestock body feature sampling. The network divides the cow into seven parts, including the body and legs. Moreover, the key points of body size are located in the different parts. It combines density measurement, point cloud slicing, contour extraction, point cloud repair, etc. A total of 137 items of cattle data are collected. Compared with some of the other models, the DUOS algorithm improves the accuracy of the segmentation task and mean intersection by 0.53% and 1.21%, respectively. Moreover, compared with the manual measurement results, the relative errors of the experimental measurement results are as follows: withers height, 1.18%; hip height, 1.34%; body length, 2.52%; thoracic circumference, 2.12%; abdominal circumference, 2.26%; and cannon circumference, 2.78%. In summary, the model is proven to have a good segmentation effect on cattle bodies and is suitable for cattle body size measurement.

RESUMEN

Robotic assembling is a challenging task that requires cognition and dexterity. In recent years, perception tools have achieved tremendous success in endowing the cognitive capabilities to robots. Although these tools have succeeded in tasks such as detection, scene segmentation, pose estimation and grasp manipulation, the associated datasets and the dataset contents lack crucial information that requires adapting them for assembling pose estimation. Furthermore, existing datasets of object 3D meshes and point clouds are presented in non-canonical view frames and therefore lack information to train perception models that infer on a visual scene. The dataset presents 2 simulated object assembly scenes with RGB-D images, 3D mesh files and ground truth assembly poses as an extension for the State-of-the-Art BOP format. This enables smooth expansion of existing perception models in computer vision as well as development of novel algorithms for estimating assembly pose in robotic assembly manipulation tasks.

RESUMEN

LiDAR point clouds can effectively depict the motion and posture of objects in three-dimensional space. Many studies accomplish the 3D object detection by voxelizing point clouds. However, in autonomous driving scenarios, the sparsity and hollowness of point clouds create some difficulties for voxel-based methods. The sparsity of point clouds makes it challenging to describe the geometric features of objects. The hollowness of point clouds poses difficulties for the aggregation of 3D features. We propose a two-stage 3D object detection framework, called MS23D. (1) We propose a method using voxel feature points from multi-branch to construct the 3D feature layer. Using voxel feature points from different branches, we construct a relatively compact 3D feature layer with rich semantic features. Additionally, we propose a distance-weighted sampling method, reducing the loss of foreground points caused by downsampling and allowing the 3D feature layer to retain more foreground points. (2) In response to the hollowness of point clouds, we predict the offsets between deep-level feature points and the object's centroid, making them as close as possible to the object's centroid. This enables the aggregation of these feature points with abundant semantic features. For feature points from shallow-level, we retain them on the object's surface to describe the geometric features of the object. To validate our approach, we evaluated its effectiveness on both the KITTI and ONCE datasets.

Asunto(s)

RESUMEN

High-precision step feature lines play a crucial role in open-pit mine design, production scheduling, mining volume calculations, road network planning, and slope maintenance. Compared with the feature lines of the geometric model, step feature lines are more irregular, complex, higher in density, and richer in detail. In this study, a novel technique for extracting step feature line from large-scale point clouds of open-pit mine by leveraging structural attributes, that is, SFLE_OPM (Step Feature Line Extraction for Open-Pit Mine), is proposed. First, we adopt the k-dimensional tree (KD-tree) resampling method to reduce the point-cloud density while retaining point-cloud features and utilize bilateral filtering for denoising. Second, we use Point Cloud Properties Network (PCPNET) to estimate the normal, calculate the slope and aspect, and then filter them. We then apply morphological operations to the step surface and obtain more continuous and smoother slope lines. In addition, we construct an Open-Pit Mine Step Feature Line (OPMSFL) dataset and benchmarked SFLE_OPM, achieving an accuracy score of 89.31% and true positive rate score of 80.18%. The results demonstrate that our method yields a higher extraction accuracy and precision than most of the existing methods. Our dataset is available at https://github.com/OPMDataSets/OPMSFL .

RESUMEN

Semantic segmentation of target objects in power transmission line corridor point cloud scenes is a crucial step in powerline tree barrier detection. The massive quantity, disordered distribution, and non-uniformity of point clouds in power transmission line corridor scenes pose significant challenges for feature extraction. Previous studies have often overlooked the core utilization of spatial information, limiting the network's ability to understand complex geometric shapes. To overcome this limitation, this paper focuses on enhancing the deep expression of spatial geometric information in segmentation networks and proposes a method called BDF-Net to improve RandLA-Net. For each input 3D point cloud data, BDF-Net first encodes the relative coordinates and relative distance information into spatial geometric feature representations through the Spatial Information Encoding block to capture the local spatial structure of the point cloud data. Subsequently, the Bilinear Pooling block effectively combines the feature information of the point cloud with the spatial geometric representation by leveraging its bilinear interaction capability thus learning more discriminative local feature descriptors. The Global Feature Extraction block captures the global structure information in the point cloud data by using the ratio between the point position and the relative position, so as to enhance the semantic understanding ability of the network. In order to verify the performance of BDF-Net, this paper constructs a dataset, PPCD, for the point cloud scenario of transmission line corridors and conducts detailed experiments on it. The experimental results show that BDF-Net achieves significant performance improvements in various evaluation metrics, specifically achieving an OA of 97.16%, a mIoU of 77.48%, and a mAcc of 87.6%, which are 3.03%, 16.23%, and 18.44% higher than RandLA-Net, respectively. Moreover, comparisons with other state-of-the-art methods also verify the superiority of BDF-Net in point cloud semantic segmentation tasks.

RESUMEN

Global pose refinement is a significant challenge within Simultaneous Localization and Mapping (SLAM) frameworks. For LIDAR-based SLAM systems, pose refinement is integral to correcting drift caused by the successive registration of 3D point clouds collected by the sensor. A divergence between the actual and calculated platform paths characterizes this error. In response to this challenge, we propose a linear, parameter-free model that uses a closed circuit for global trajectory corrections. Our model maps rotations to quaternions and uses Spherical Linear Interpolation (SLERP) for transitions between them. The intervals are established by the constraint set by the Least Squares (LS) method on rotation closure and are proportional to the circuit's size. Translations are globally adjusted in a distinct linear phase. Additionally, we suggest a coarse-to-fine pairwise registration method, integrating Fast Global Registration and Generalized ICP with multiscale sampling and filtering. The proposed approach is tested on three varied datasets of point clouds, including Mobile Laser Scanners and Terrestrial Laser Scanners. These diverse datasets affirm the model's effectiveness in 3D pose estimation, with substantial pose differences and efficient pose optimization in larger circuits.

RESUMEN

Understanding geometric and biophysical characteristics is essential for determining grapevine vigor and improving input management and automation in viticulture. This study compares point cloud data obtained from a Terrestrial Laser Scanner (TLS) and various UAV sensors including multispectral, panchromatic, Thermal Infrared (TIR), RGB, and LiDAR data, to estimate geometric parameters of grapevines. Descriptive statistics, linear correlations, significance using the F-test of overall significance, and box plots were used for analysis. The results indicate that 3D point clouds from these sensors can accurately estimate maximum grapevine height, projected area, and volume, though with varying degrees of accuracy. The TLS data showed the highest correlation with grapevine height (r = 0.95, p < 0.001; R2 = 0.90; RMSE = 0.027 m), while point cloud data from panchromatic, RGB, and multispectral sensors also performed well, closely matching TLS and measured values (r > 0.83, p < 0.001; R2 > 0.70; RMSE < 0.084 m). In contrast, TIR point cloud data performed poorly in estimating grapevine height (r = 0.76, p < 0.001; R2 = 0.58; RMSE = 0.147 m) and projected area (r = 0.82, p < 0.001; R2 = 0.66; RMSE = 0.165 m). The greater variability observed in projected area and volume from UAV sensors is related to the low point density associated with spatial resolution. These findings are valuable for both researchers and winegrowers, as they support the optimization of TLS and UAV sensors for precision viticulture, providing a basis for further research and helping farmers select appropriate technologies for crop monitoring.

RESUMEN

This paper presents a novel segmentation algorithm specially developed for applications in 3D point clouds with high variability and noise, particularly suitable for heritage building 3D data. The method can be categorized within the segmentation procedures based on edge detection. In addition, it uses a graph-based topological structure generated from the supervoxelization of the 3D point clouds, which is used to make the closure of the edge points and to define the different segments. The algorithm provides a valuable tool for generating results that can be used in subsequent classification tasks and broader computer applications dealing with 3D point clouds. One of the characteristics of this segmentation method is that it is unsupervised, which makes it particularly advantageous for heritage applications where labelled data is scarce. It is also easily adaptable to different edge point detection and supervoxelization algorithms. Finally, the results show that the 3D data can be segmented into different architectural elements, which is important for further classification or recognition. Extensive testing on real data from historic buildings demonstrated the effectiveness of the method. The results show superior performance compared to three other segmentation methods, both globally and in the segmentation of planar and curved zones of historic buildings.

RESUMEN

In order to guide orchard management robots to realize some tasks in orchard production such as autonomic navigation and precision spraying, this research proposed a deep-learning network called dynamic fusion segmentation network (DFSNet). The network contains a local feature aggregation (LFA) layer and a dynamic fusion segmentation architecture. The LFA layer uses the positional encoders for initial transforming embedding, and progressively aggregates local patterns via the multi-stage hierarchy. The fusion segmentation module (Fus-Seg) can format point tags by learning a multi-embedding space, and the generated tags can further mine the point cloud features. At the experimental stage, significant segmentation results of the DFSNet were demonstrated on the dataset of orchard fields, achieving an accuracy rate of 89.43% and an mIoU rate of 74.05%. DFSNet outperforms other semantic segmentation networks, such as PointNet, PointNet++, D-PointNet++, DGCNN, and Point-NN, with improved accuracies over them by 11.73%, 3.76%, 2.36%, and 2.74%, respectively, and improved mIoUs over the these networks by 28.19%, 9.89%, 6.33%, 9.89, and 24.69%, respectively, on the all-scale dataset (simple-scale dataset + complex-scale dataset). The proposed DFSNet can capture more information from orchard scene point clouds and provide more accurate point cloud segmentation results, which are beneficial to the management of orchards.

RESUMEN

This paper proposes an autonomous robotic system to prune sweet pepper leaves using semantic segmentation with deep learning and an articulated manipulator. This system involves three main tasks: the perception of crop parts, the detection of pruning position, and the control of the articulated manipulator. A semantic segmentation neural network is employed to recognize the different parts of the sweet pepper plant, which is then used to create 3D point clouds for detecting the pruning position and the manipulator pose. Eventually, a manipulator robot is controlled to prune the crop part. This article provides a detailed description of the three tasks involved in building the sweet pepper pruning system and how to integrate them. In the experiments, we used a robot arm to manipulate the pruning leaf actions within a certain height range and a depth camera to obtain 3D point clouds. The control program was developed in different modules using various programming languages running on the ROS (Robot Operating System).

RESUMEN

Generative models have the potential to revolutionize 3D extended reality. A primary obstacle is that augmented and virtual reality need real-time computing. Current state-of-the-art point cloud random generation methods are not fast enough for these applications. We introduce a vector-quantized variational autoencoder model (VQVAE) that can synthesize high-quality point clouds in milliseconds. Unlike previous work in VQVAEs, our model offers a compact sample representation suitable for conditional generation and data exploration with potential applications in rapid prototyping. We achieve this result by combining architectural improvements with an innovative approach for probabilistic random generation. First, we rethink current parallel point cloud autoencoder structures, and we propose several solutions to improve robustness, efficiency and reconstruction quality. Notable contributions in the decoder architecture include an innovative computation layer to process the shape semantic information, an attention mechanism that helps the model focus on different areas and a filter to cover possible sampling errors. Secondly, we introduce a parallel sampling strategy for VQVAE models consisting of a double encoding system, where a variational autoencoder learns how to generate the complex discrete distribution of the VQVAE, not only allowing quick inference but also describing the shape with a few global variables. We compare the proposed decoder and our VQVAE model with established and concurrent work, and we prove, one by one, the validity of the single contributions.

RESUMEN

Light Detection and Ranging (LiDAR) is a well-established active technology for the direct acquisition of 3D data. In recent years, the geometric information collected by LiDAR sensors has been widely combined with optical images to provide supplementary spectral information to achieve more precise results in diverse remote sensing applications. The emergence of active Multispectral LiDAR (MSL) systems, which operate on different wavelengths, has recently been revolutionizing the simultaneous acquisition of height and intensity information. So far, MSL technology has been successfully applied for fine-scale mapping in various domains. However, a comprehensive review of this modern technology is currently lacking. Hence, this study presents an exhaustive overview of the current state-of-the-art in MSL systems by reviewing the latest technologies for MSL data acquisition. Moreover, the paper reports an in-depth analysis of the diverse applications of MSL, spanning across fields of "ecology and forestry", "objects and Land Use Land Cover (LULC) classification", "change detection", "bathymetry", "topographic mapping", "archaeology and geology", and "navigation". Our systematic review uncovers the potentials, opportunities, and challenges of the recently emerged MSL systems, which integrate spatial-spectral data and unlock the capability for precise multi-dimensional (nD) mapping using only a single-data source.

RESUMEN

Spray drift is inevitable in chemical applications, drawing global attention because of its potential environmental pollution and the risk of exposing bystanders to pesticides. This issue has become more pronounced with a growing consensus on the need for enhanced environmental safeguards in agricultural practices. Traditionally, spray drift measurements, crucial for refining spray techniques, relied on intricate, time-consuming, and labor-intensive sampling methods utilizing passive collectors. In this study, we investigated the feasibility of using close-range remote sensing technology based on Light Detection and Ranging (LiDAR) point clouds to implement drift measurements and drift reduction classification. The results show that LiDAR-based point clouds vividly depict the spatial dispersion and movement of droplets within the vertical plane. The capability of LiDAR to accurately determine drift deposition was demonstrated, evident from the high R2 values of 0.847, 0.748 and 0.860 achieved for indoor, wind tunnel and field environments, respectively. Droplets smaller than 100 µm and with a density below 50 deposits·cm-2·s-1 posed challenges for LiDAR detection. To address these challenges, the use of multichannel LiDAR with higher wavelengths presents a potential solution, warranting further exploration. Furthermore, we found a satisfactory consistency when comparing the drift reduction classification calculated from LiDAR measurements with those obtained though passive collectors, both in indoor tests and the unmanned air-assisted sprayer (UAAS) field test. However, in environments with less dense clouds of larger droplets, a contradiction emerged between higher drift deposition and lower scanned droplet counts, potentially leading to deviations in the calculated drift potential reduction percentage (DPRP). This was exemplified in a field test using an unmanned aerial vehicle sprayer (UAVS). Our findings provide valuable insights into the monitoring and quantification of pesticide drift at close range using LiDAR technology, paving the way for more precise and efficient drift assessment methodologies.

RESUMEN

The quality of chiral environment (i.e. catalytic pocket) is directly related to the performance of chiral catalysts. The existing methods need super computing power and time, i.e., it is difficult to quickly judge the interaction between chiral catalysts and substrates for accurately evaluating the effects of chiral catalytic pockets. In this paper, for the 3D isosurface point clouds of molecular electrostatic potential, by using computer simulations, we propose a robust method to detect interacted points, and then accurately have the corresponding interacted volumes. First, by using the existing marching cubes algorithm, we construct the 3D models with triangular surface for isosurface point clouds of molecular electrostatic potentials. Second, by using our improved hierarchical bounding boxes algorithm, we significantly filter out most redundant non-collision points. Third, by using the normal vectors of the remaining points and related triangles, we robustly determine the interacted points to construct interacted sets. And finally, by combining the classical slicing with our multi-contour segmenting, we accurately calculate the interacted volumes. Over three groups of the point clouds of the chemical molecules, experimental results show that our method effectively removes the non-interacted points at average rates of 71.65%, 77.76%, and 71.82%, and calculates the interacted volumes with the average relative errors of 1.7%, 1.6%, and 1.9%, respectively.

Asunto(s)

RESUMEN

In this study, we introduce Blob-B-Gone, a lightweight framework to computationally differentiate and eventually remove dense isotropic localization accumulations (blobs) caused by artifactually immobilized particles in MINFLUX single-particle tracking (SPT) measurements. This approach uses purely geometrical features extracted from MINFLUX-detected single-particle trajectories, which are treated as point clouds of localizations. Employing k-means++ clustering, we perform single-shot separation of the feature space to rapidly extract blobs from the dataset without the need for training. We automatically annotate the resulting sub-sets and, finally, evaluate our results by means of principal component analysis (PCA), highlighting a clear separation in the feature space. We demonstrate our approach using two- and three-dimensional simulations of freely diffusing particles and blob artifacts based on parameters extracted from hand-labeled MINFLUX tracking data of fixed 23-nm bead samples and two-dimensional diffusing quantum dots on model lipid membranes. Applying Blob-B-Gone, we achieve a clear distinction between blob-like and other trajectories, represented in F1 scores of 0.998 (2D) and 1.0 (3D) as well as 0.995 (balanced) and 0.994 (imbalanced). This framework can be straightforwardly applied to similar situations, where discerning between blob and elongated time traces is desirable. Given a number of localizations sufficient to express geometric features, the method can operate on any generic point clouds presented to it, regardless of its origin.

RESUMEN

Object detection is important in many applications, such as autonomous driving. While 2D images lack depth information and are sensitive to environmental conditions, 3D point clouds can provide accurate depth information and a more descriptive environment. However, sparsity is always a challenge in single-frame point cloud object detection. This paper introduces a two-stage proposal-based feature fusion method for object detection using multiple frames. The proposed method, called proposal features fusion (PFF), utilizes a cosine-similarity approach to associate proposals from multiple frames and employs an attention weighted fusion (AWF) module to merge features from these proposals. It allows for feature fusion specific to individual objects and offers lower computational complexity while achieving higher precision. The experimental results on the nuScenes dataset demonstrate the effectiveness of our approach, achieving an mAP of 46.7%, which is 1.3% higher than the state-of-the-art 3D object detection method.

RESUMEN

In interpreting a scene for numerous applications, including autonomous driving and robotic navigation, semantic segmentation is crucial. Compared to single-modal data, multi-modal data allow us to extract a richer set of features, which is the benefit of improving segmentation accuracy and effect. We propose a point cloud semantic segmentation method, and a fusion graph convolutional network (FGCN) which extracts the semantic information of each point involved in the two-modal data of images and point clouds. The two-channel k-nearest neighbors (KNN) module of the FGCN was created to address the issue of the feature extraction's poor efficiency by utilizing picture data. Notably, the FGCN utilizes the spatial attention mechanism to better distinguish more important features and fuses multi-scale features to enhance the generalization capability of the network and increase the accuracy of the semantic segmentation. In the experiment, a self-made semantic segmentation KITTI (SSKIT) dataset was made for the fusion effect. The mean intersection over union (MIoU) of the SSKIT can reach 88.06%. As well as the public datasets, the S3DIS showed that our method can enhance data features and outperform other methods: the MIoU of the S3DIS can reach up to 78.55%. The segmentation accuracy is significantly improved compared with the existing methods, which verifies the effectiveness of the improved algorithms.

RESUMEN

Cervical cancer poses a serious threat to the health of women and radiotherapy is one of the primary treatment methods for this condition. However, this treatment is associated with a high risk of causing acute hematologic toxicity. Delineating the bone marrow (BM) for sparing based on computer tomography (CT) images before radiotherapy can effectively avoid this risk. Unfortunately, compared to magnetic resonance (MR) images, CT images lack the ability to express the activity of BM. Therefore, medical practitioners currently manually delineate the BM on CT images by corresponding to MR images. However, the manual delineation of BM is time-consuming and cannot guarantee accuracy due to the inconsistency of the CT-MR multimodal images. This study proposes a multimodal image-oriented automatic registration method for pelvic BM sparing. The proposed method includes three-dimensional (3D) bone point clouds reconstruction and an iterative closest point registration based on a local spherical system for marking BM on CT images. By introducing a joint coordinate system that combines the global Cartesian coordinate system with the local point clouds' spherical coordinate system, the increasement of point descriptive dimension avoids the local optimal registration and improves the registration accuracy. Experiments on the dataset of patients demonstrate that our proposed method can enhance the multimodal image registration accuracy and efficiency for medical practitioners in BM-sparing of cervical cancer radiotherapy. The method proposed in this contribution might also provide a solution to multimodal registration, especially in multimodal sequential images in other clinical applications, such as the diagnosis of cervical cancer and the preservation of normal organs during radiotherapy.

RESUMEN

Three-dimensional face recognition is an important part of the field of computer vision. Point clouds are widely used in the field of 3D vision due to the simple mathematical expression. However, the disorder of the points makes it difficult for them to have ordered indexes in convolutional neural networks. In addition, the point clouds lack detailed textures, which makes the facial features easily affected by expression or head pose changes. To solve the above problems, this paper constructs a new face recognition network, which mainly consists of two parts. The first part is a novel operator based on a local feature descriptor to realize the fine-grained features extraction and the permutation invariance of point clouds. The second part is a feature enhancement mechanism to enhance the discrimination of facial features. In order to verify the performance of our method, we conducted experiments on three public datasets: CASIA-3D, Bosphorus, and Lock3Dface. The results show that the accuracy of our method is improved by 0.7%, 0.4%, and 0.8% compared with the latest methods on these three datasets, respectively.