Arsenic contamination causes numerous health problems for humans and wildlife via bioaccumulation in the food chain. Phytoremediation of arsenic-contaminated soils with the model arsenic hyperaccumulator Pteris vittata provides a promising way to reduce the risk, in which the growth and arsenic absorption ability of plants and the biotransformation of soil arsenic may be greatly affected by rhizosphere microorganisms. However, the microbial community composition in the rhizosphere of P. vittata and its functional role in arsenic phytoremediation are still poorly understood. To bridge this knowledge gap, we carried out a field investigation and pot experiment to explore the composition and functional implications of microbial communities in the rhizosphere of four P. vittata populations with a natural arsenic contamination gradient. Arsenic pollution significantly reduced bacterial and fungal diversity in the rhizosphere of P. vittata (p < 0.05) and played an important role in shaping the microbial community structure. The suitability of soil microbes for the growth of P. vittata gradually decreased following increased soil arsenic levels, as indicated by the increased abundance of pathogenic fungi and parasitic bacteria and the decrease in symbiotic fungi. The analysis of arsenic-related functional gene abundance with AsChip revealed the gradual enrichment of the microbial genes involved in As(III) oxidation, As(V) reduction, and arsenic methylation and demethylation in the rhizosphere of P. vittata following increased arsenic levels (p < 0.05). The regulation of indigenous soil microbes through the field application of fungicide, but not bactericide, significantly reduced the remediation efficiency of P. vittata grown under an arsenic contamination gradient, indicating the important role of indigenous fungal groups in the remediation of arsenic-contaminated soil. This study has important implications for the functional role and application prospects of soil microorganisms in the phytoremediation of arsenic-polluted soil.