Subjective assessment of emotional valence is typically associated with both brain activity and autonomic arousal. Accurately assessing emotional salience is particularly important when perceiving threat. We sought to characterize the neural correlates of the interaction between behavioral and autonomic responses to potentially threatening visual and auditory stimuli. Twenty-five healthy male subjects underwent fMRI scanning whilst skin conductance responses (SCR) were recorded. One hundred and eighty pictures, sentences, and sounds were assessed as "harmless" or "threatening." Individuals' stimulus-locked, phasic SCRs and trial-by-trial behavioral assessments were entered as regressors into a flexible factorial design to establish their separate autonomic and behavioral neural correlates, and convolved to examine psycho-autonomic interaction (PAI) effects. Across all stimuli, "threatening," compared with "harmless" behavioral assessments were associated with mainly frontal and precuneus activation with specific within-modality activations including bilateral parahippocampal gyri (pictures), bilateral anterior cingulate cortex (ACC) and frontal pole (sentences), and right Heschl's gyrus and bilateral temporal gyri (sounds). Across stimulus modalities SCRs were associated with activation of parieto-occipito-thalamic regions, an activation pattern which was largely replicated within-modality. In contrast, PAI analyses revealed modality-specific activations including right fusiform/parahippocampal gyrus (pictures), right insula (sentences), and mid-cingulate gyrus (sounds). Phasic SCR activity was positively correlated with an individual's propensity to assess stimuli as "threatening." SCRs may modulate cognitive assessments on a "harmless-threatening" dimension, thereby modulating affective tone and hence behavior.