OBJECTIVES: Outcome of endovascular treatment in acute ischemic stroke patients depends on collateral circulation to provide blood supply to the ischemic territory. We evaluated the performance of a commercially available algorithm for assessing the collateral score (CS) in acute ischemic stroke patients. METHODS: Retrospectively, baseline CTA scans (≤ 3-mm slice thickness) with an intracranial carotid artery (ICA), middle cerebral artery segment M1 or M2 occlusion, from the MR CLEAN Registry (n = 1627) were evaluated. All CTA scans were evaluated for visual CS (0-3) by eight expert radiologists (reference standard). A Web-based AI algorithm quantified the collateral circulation (0-100%) for correctly detected occlusion sides. Agreement between visual CS and categorized automated CS (0: 0%, 1: > 0- ≤ 50%, 2: > 50- < 100%, 3: 100%) was assessed. Area under the curve (AUC) values for classifying patients in having good (CS: 2-3) versus poor (CS: 0-1) collaterals and for predicting functional independence (90-day modified Rankin Scale 0-2) were computed. Influence of CTA acquisition timing after contrast material administration was reported. RESULTS: In the analyzed scans (n = 1024), 59% agreement was found between visual CS and automated CS. An AUC of 0.87 (95% CI: 0.85-0.90) was found for discriminating good versus poor CS. Timing of CTA acquisition did not influence discriminatory performance. AUC for predicting functional independence was 0.66 (95% CI 0.62-0.69) for automated CS, similar to visual CS 0.64 (95% CI 0.61-0.68). CONCLUSIONS: The automated CS performs similar to radiologists in determining a good versus poor collateral score and predicting functional independence in acute ischemic stroke patients with a large vessel occlusion. KEY POINTS: • Software for automated quantification of intracerebral collateral circulation on computed tomography angiography performs similar to expert radiologists in determining a good versus poor collateral score. • Software for automated quantification of intracerebral collateral circulation on computed tomography angiography performs similar to expert radiologists in predicting functional independence in acute ischemic stroke patients with a large vessel occlusion. • The timing of computed tomography angiography acquisition after contrast material administration did not influence the performance of automated quantification of the collateral status.