Adaptively managing marine protected areas (MPAs) requires accurately assessing whether established MPAs are achieving their goals of protecting and conserving biomass, especially for harvested populations. Ecological MPA assessments commonly compare inside of the MPA to a reference point outside of and/or before implementation (i.e., calculating "response ratios"). Yet, MPAs are not simple ecological experiments; by design, protected populations interact with those outside, and population dynamic responses can be nonlinear. This complicates assessment interpretations. Here, we used a two-patch population model to explore how MPA response ratios (outside-inside, before-after, and before-after-control-impact [BACI]) for fished populations behave under different conditions, like whether the population is receiving a sustainable larval supply or if it is declining despite protection from harvest. We then conducted a Bayesian evaluation of MPA effects on fish and invertebrate populations based on data collected from 82 published studies on 264 no-take MPAs worldwide, using the results of an earlier global meta-analysis as priors. We considered the effects of calculating different summary metrics on these results, drawing on the theoretical insights from our population model as a comparative framework. We demonstrate that not all response ratio comparison types provide the same information For example, outside-inside and BACI comparisons can fail to detect population decline within MPAs, whereas before-after comparisons likely detect that pattern. Considering these limitations, we nonetheless found that MPAs globally are producing positive outcomes, with on average greater biomass, density, and organism size within their boundaries than reference sites. However, only a small portion of studies (18 of 82) provided the temporal data necessary to determine that protection, on average, has led to increased abundance of populations within MPAs over time. These findings demonstrate the importance of considering the underlying system dynamics when assessing MPA effects. Assuming that large outside-inside or BACI response ratios always reflect large and net positive conservation effects may lead to misleading conclusions, we recommend that (1) when assessing specific MPA effects, empirical findings be considered alongside theoretical knowledge relevant to that MPA system, and (2) management should respond to the local conditions and outcomes, rather than a blanket expectation for positive MPA effects.