Alfalfa (Medicago sativa L.) silage (AS) is an important feedstuff in ruminant nutrition. However, its high non-protein nitrogen content often leads to poor ruminal nitrogen retention. Various pre-ensiling treatments differing with respect to dry matter concentrations, wilting intensities and sucrose addition have been previously shown to improve the quality and true protein preservation of AS, and have substantial effects on in vitro ruminal fermentation of the resulting silages. However, it is unknown how these pre-ensiling treatments affect the ruminal microbiota composition, and whether alterations in the microbiota explain previously observed differences in ruminal fermentation. Therefore, during AS incubation in a rumen simulation system, liquid and solid phases were sampled 2 and 7 days after first incubating AS, representing an early (ET) and late (LT) time point, respectively. Subsequently, DNA was extracted and qPCR (bacteria, archaea, and anaerobic fungi) and prokaryotic 16S rRNA gene amplicon sequence analyses were performed. At the ET, high dry matter concentration and sucrose addition increased concentrations of archaea in the liquid phase (P = 0.001) and anaerobic fungi in the solid phase (P < 0.001). At the LT, only sucrose addition increased archaeal concentration in the liquid phase (P = 0.014) and anaerobic fungal concentration in the solid phase (P < 0.001). Bacterial concentrations were not affected by pre-ensiling treatments. The prokaryotic phylogenetic diversity index decreased in the liquid phase from ET to LT (P = 0.034), whereas the solid phase was not affected (P = 0.060). This is suggestive of a general adaption of the microbiota to the soluble metabolites released from the incubated AS, particularly regarding the sucrose-treated AS. Redundancy analysis of the sequence data at the genus level indicated that sucrose addition (P = 0.001), time point (P = 0.001), and their interaction (P = 0.001) affected microbial community composition in both phases. In summary, of the pre-ensiling treatments tested sucrose addition had the largest effect on the microbiota, and together with sampling time point affected microbiota composition in both phases of the rumen simulation system. Thus, microbiota composition analysis helped to understand the ruminal fermentation patterns, but could not fully explain them.