BACKGROUND: Functional neuroimaging studies have identified a set of nodes in the occipital-temporal cortex that preferentially respond to faces in comparison with other visual objects. By contrast, the processing of places seems to rely on parahippocampal cortex and structures heavily implicated in memory (e.g., the hippocampus). It has been suggested that human aging leads to decreased neural specialization of core face and place processing areas and impairments in face and place perception. METHODS: Using mediation analysis, we tested the potential contribution of micro- and macrostructure within the hippocampal and occipitotemporal systems to age-associated effects in face and place category processing (as measured by 1-back working memory tasks) in 55 healthy adults (age range 23-79 years). To test for specific contributions of the studied structures to face/place processing, we also studied a distinct tract (i.e., the anterior thalamic radiation [ATR]) and cognitive performance for other visual object categories (objects, bodies, and verbal material). Constrained spherical deconvolution-based tractography was used to reconstruct the fornix, the inferior longitudinal fasciculus (ILF), and the ATR. Hippocampal volumetric measures were segmented from FSL-FIRST toolbox. RESULTS: It was found that age associates with (a) decreases in fractional anisotropy (FA) in the fornix, in right ILF (but not left ILF), and in the ATR (b) reduced volume in the right and left hippocampus and (c) decline in visual object category processing. Importantly, mediation analysis showed that micro- and macrostructural impairments in the fornix and right hippocampus, respectively, associated with age-dependent decline in place processing. Alternatively, microstructural impairments in right hemispheric ILF associated with age-dependent decline in face processing. There were no other mediator effects of micro- and macrostructural variables on age-cognition relationships. CONCLUSION: Together, the findings support specific contributions of the fornix and right hippocampus in visuospatial scene processing and of the long-range right hemispheric occipitotemporal network in face category processing.