The concept of ultra-high risk for psychosis (UHR) has been at the forefront of psychiatric research for several decades, with the ultimate goal of preventing the onset of psychotic disorder in high-risk individuals. Orygen (Melbourne, Australia) has led a range of observational and intervention studies in this clinical population. These datasets have now been integrated into the UHR 1000+ cohort, consisting of a sample of 1,245 UHR individuals with a follow-up period ranging from 1 to 16.7 years. This paper describes the cohort, presents a clinical prediction model of transition to psychosis in this cohort, and examines how predictive performance is affected by changes in UHR samples over time. We analyzed transition to psychosis using a Cox proportional hazards model. Clinical predictors for transition to psychosis were investigated in the entire cohort using multiple imputation and Rubin's rule. To assess performance drift over time, data from 1995-2016 were used for initial model fitting, and models were subsequently validated on data from 2017-2020. Over the follow-up period, 220 cases (17.7%) developed a psychotic disorder. Pooled hazard ratio (HR) estimates showed that the Comprehensive Assessment of At-Risk Mental States (CAARMS) Disorganized Speech subscale severity score (HR=1.12, 95% CI 1.02-1.24, p=0.024), the CAARMS Unusual Thought Content subscale severity score (HR=1.13, 95% CI 1.03-1.24, p=0.009), the Scale for the Assessment of Negative Symptoms (SANS) total score (HR=1.02, 95% CI 1.00-1.03, p=0.022), the Social and Occupational Functioning Assessment Scale (SOFAS) score (HR=0.98, 95% CI 0.97-1.00, p=0.036), and time between onset of symptoms and entry to UHR service (log transformed) (HR=1.10, 95% CI 1.02-1.19, p=0.013) were predictive of transition to psychosis. UHR individuals who met the brief limited intermittent psychotic symptoms (BLIPS) criteria had a higher probability of transitioning to psychosis than those who met the attenuated psychotic symptoms (APS) criteria (HR=0.48, 95% CI 0.32-0.73, p=0.001) and those who met the Trait risk criteria (a first-degree relative with a psychotic disorder or a schizotypal personality disorder plus a significant decrease in functioning during the previous year) (HR=0.43, 95% CI 0.22-0.83, p=0.013). Models based on data from 1995-2016 displayed good calibration at initial model fitting, but showed a drift of 20.2-35.4% in calibration when validated on data from 2017-2020. Large-scale longitudinal data such as those from the UHR 1000+ cohort are required to develop accurate psychosis prediction models. It is critical to assess existing and future risk calculators for temporal drift, that may reduce their utility in clinical practice over time.