RESUMEN
With the popularization of big data technology, agricultural data processing systems have become more intelligent. In this study, a data processing method for farmland environmental monitoring based on improved Spark components is designed. It introduces the FAST-Join (Join critical filtering sampling partition optimization) algorithm in the Spark component for equivalence association query optimization to improve the operating efficiency of the Spark component and cluster. The experimental results show that the amount of data written and read in Shuffle by Spark optimized by the FAST-join algorithm only accounts for 0.958 and 1.384% of the original data volume on average, and the calculation speed is 202.11% faster than the original. The average data processing time and occupied memory size of the Spark cluster are reduced by 128.22 and 76.75% compared with the originals. It also compared the cluster performance of the FAST-join and Equi-join algorithms. The Spark cluster optimized by the FAST-join algorithm reduced the processing time and occupied memory size by an average of 68.74 and 37.80% compared with the Equi-join algorithm, which shows that the FAST-join algorithm can effectively improve the efficiency of inter-data table querying and cluster computing.
RESUMEN
This technical note critically evaluates the transformative potential of Artificial Intelligence (AI) and sensor technologies in the swiftly evolving dairy livestock export industry. We focus on the novel application of the Internet of Things (IoT) in long-distance livestock transportation, particularly in livestock enumeration and identification for precise traceability. Technological advancements in identifying behavioral patterns in 'shy feeder' cows and real-time weight monitoring enhance the accuracy of long-haul livestock transportation. These innovations offer benefits such as improved animal welfare standards, reduced supply chain inaccuracies, and increased operational productivity, expanding market access and enhancing global competitiveness. However, these technologies present challenges, including individual animal customization, economic analysis, data security, privacy, technological adaptability, training, stakeholder engagement, and sustainability concerns. These challenges intertwine with broader ethical considerations around animal treatment, data misuse, and the environmental impacts. By providing a strategic framework for successful technology integration, we emphasize the importance of continuous adaptation and learning. This note underscores the potential of AI, IoT, and sensor technologies to shape the future of the dairy livestock export industry, contributing to a more sustainable and efficient global dairy sector.