The purpose of this study is to clarify the difference in oil production rules of conglomerate reservoirs with different pore structures during surfactant-polymer (SP) binary flooding and to ensure the efficient development of conglomerate reservoirs. In this paper, the full-diameter natural cores from the conglomerate reservoir of the Triassic Kexia Formation in the seventh middle block of the Karamay Oilfield (Xinjiang, China) are selected as the research objects. Two schemes of single constant viscosity (SCV) and echelon viscosity reducing (EVR) are designed to displace oil from three main oil-bearing lithologies, namely fine conglomerate, glutenite, and sandstone. Through comprehensive analysis of parameters, such as oil recovery rate, water content, and injection pressure difference, the influence of lithology on the enhanced oil recovery (EOR) of the EVR scheme is determined, which in turn reveals the differences in the step-wise oil production rules of the three lithologies. The experimental results show that for the three lithological reservoirs, the oil displacement effect of the EVR scheme is better than that of the SCV scheme, and the differences in recovery rates between the two schemes are 9.91% for the fine conglomerate, 6.77% for glutenite, and 6.69% for sandstone. By reducing the molecular weight and viscosity of the SP binary system, the SCV scheme achieves the reconstruction of the pressure field and the redistribution of seepage paths of chemical micelles with different sizes, thus, achieving the step-wise production of crude oil in different scale pore throats and enhancing the overall recovery of the reservoir. The sedimentary environment and diagenesis of the three types of lithologies differ greatly, resulting in diverse microscopic pore structures and differential seepage paths and displace rules of SP binary solutions, ultimately leading to large differences in the enhanced oil recoveries of different lithologies. The fine conglomerate reservoir has the strongest anisotropy, the worst pore throat connectivity, and the lowest water flooding recovery rate. Since the fine conglomerate reservoir has the strongest anisotropy, the worst pore throats connectivity, and the lowest water flooding recovery, the EVR scheme shows a good "water control and oil enhancement" development feature and the best step-wise oil production effect. The oil recovery rate of the two schemes for fine conglomerate shows a difference of 10.14%, followed by 6.36% for glutenite and 5.10% for sandstone. In addition, the EOR of fine conglomerate maintains a high upward trend throughout the chemical flooding, indicating that the swept volume of small pore throats gradually expands and the producing degree of the remaining oil in it gradually increases. Therefore, the fine conglomerate is the most suitable lithology for the SCV scheme among the three lithologies of the conglomerate reservoirs.