Women's empowerment is a process that includes increases in intrinsic agency (power within); instrumental agency (power to); and collective agency (power with). We used baseline data from two studies-Targeting and Realigning Agriculture for Improved Nutrition (TRAIN) in Bangladesh and Building Resilience in Burkina Faso (BRB)-to assess the measurement properties of survey questions operationalizing selected dimensions of intrinsic, instrumental, and collective agency in the project-level Women's Empowerment in Agricultural Index (pro-WEAI). We applied unidimensional item-response models to question (item) sets to assess their measurement properties, and when possible, their cross-context measurement equivalence-a requirement of measures designed for cross-group comparisons. For intrinsic agency in the right to bodily integrity, measured with five attitudinal questions about intimate partner violence (IPV) against women, model assumptions of unidimensionality and local independence were met. Four items showed good model fit and measurement equivalence across TRAIN and BRB. For item sets designed to capture autonomy in income, intrinsic agency in livelihoods activities, and instrumental agency in: livelihoods activities, the sale or use of outputs, the use of income, and borrowing from financial services, model assumptions were not met, model fit was poor, and items generally were weakly related to the latent (unobserved) agency construct. For intrinsic and instrumental agency in livelihoods activities and for instrumental agency in the sale or use of outputs and in the use of income, items sets had similar precision along the latent-agency continuum, suggesting that similar item sets could be dropped without a loss of precision. IRT models for collective agency were not estimable because of low reported presence and membership in community groups. This analysis demonstrates the use of IRT methods to assess the measurement properties of item sets in pro-WEAI, and empowerment scales generally. Findings suggest that a shorter version of pro-WEAI can be developed that will improve its measurement properties. We recommend revisions to the pro-WEAI questionnaire and call for new measures of women's collective agency.