A substantial body of evidence links differences in brain size to differences in brain organization. We have hypothesized that the developmental aspect of this relation plays a role in autism spectrum disorder (ASD), a neurodevelopmental disorder which involves abnormalities in brain growth. Children with ASD have abnormally large brains by the second year of life, and for several years thereafter their brain size can be multiple standard deviations above the norm. The greater conduction delays and cellular costs presumably associated with the longer long-distance connections in these larger brains is thought to influence developmental processes, giving rise to an altered brain organization with less communication between spatially distant regions. This has been supported by computational models and by findings linking greater intra-cranial volume, an index of maximum brain-size during development, to reduced inter-hemispheric connectivity in individuals with ASD. In this paper, we further assess this hypothesis via a whole-brain analysis of network efficiency. We utilize diffusion tractography to estimate the strength and length of the connections between all pairs of cortical regions. We compute the efficiency of communication between each network node and all others, and within local neighborhoods; we then assess the relation of these measures to intra-cranial volume, and the differences in these measures between adults with autism and typical controls. Intra-cranial volume is shown to be inversely related to efficiency for wide-spread regions of cortex. Moreover, the spatial patterns of reductions in efficiency in autism bear a striking resemblance to the regional relationships between efficiency and intra-cranial volume, particularly for local efficiency. The results thus provide further support for the hypothesized link between brain overgrowth in children with autism and the efficiency of the organization of the brain in adults with autism.