The removal of antibiotic resistance genes (ARGs) in sewage is of great concern, but advanced sewage treatment technologies are not suitable for rural areas, so the multi-layer soil infiltration system (MSL) has been developed for rural sewage treatment. However, little is known about the performance and function of MSL in the treatment of ARGs in rural sewage. Here, we optimized the matrix composition and structure of MSL and explored the efficacy and mechanism of MSL systems for ARG removal under different hydraulic conditions. The ARGs removal rate of MSL ranged from 41.51% to 99.67%, in which MSL with the middle hydraulic load, high pollution load, and continuous inflowing conditions showed the best removal performance. In addition, this system can operate stably and resist the temperature fluctuation, which showed an equivalent removal rate of ARGs in warm and cold seasons, amounting to 69.0%. The structural equation model revealed that microorganisms in sewage could significantly affect ARG removal (path coefficient = 0.91), probably owing to their interspecies competition. As for the internal system, the reduction of ARGs was mainly driven by microorganisms in the system matrix (path coefficient = 0.685), especially soil-mixture-block (SMB) microorganisms. The physicochemical factors of the matrix indirectly reduce ARGs by affecting the microorganisms that adhere to the matrices. Note that the pairwise alignment of nucleotide analysis demonstrated that the system matrix, especially biochar in the SMB, adsorbed ARGs and their hosts from the sewage, and in turn eliminated them by inhibiting the spread and colonization of hosts, thereby reducing the abundance of ARGs. Collectively, this study provides a deeper insight into the removal of ARGs from rural sewage by MSL, which can help improve sewage treatment technologies.