Cyber-physical systems (CPSs) are generally considered to be the next generation of engineered systems. However, the actual application of CPSs in the Architecture, Engineering and Construction (AEC) industry is still at a low level. The sensing method in the construction process plays a very important role in the establishment of CPSs. Therefore, the purpose of this paper is to discuss the application potential of computer vision-based sensing methods and provide practical suggestions through a literature review. This paper provides a review of the current application of CPSs in the AEC industry, summarizes the current knowledge gaps, and discusses the problems with the current construction site sensing approach. Considering the unique advantages of the computer vision (CV) method at the construction site, the application of CV for different construction entities was reviewed and summarized to achieve a CV-based construction site sensing approach for construction process CPSs. The potential of CPS can be further stimulated by providing rich information from on-site sensing using CV methods. According to the review, this approach has unique advantages in the specific environment of the construction site. Based on the current knowledge gap identified in the literature review, this paper proposes a novel concept of visual-based construction site sensing method for CPS application, and an architecture for CV-based CPS is proposed as an implementation of this concept. The main contribution of this paper is to propose a CPS architecture using computer vision as the main information acquisition method based on the literature review. This architecture innovatively introduces computer vision as a sensing method of construction sites, and realizes low-cost and non-invasive information acquisition in complex construction scenarios. This method can be used as an important supplement to on-site sensing to further promote the automation and intelligence of the construction process.