Daily situations involve many objects and behaviors. To comprehend the meaning of situations, the relationships between objects, behaviors, and the situational context are important. To reveal the cortical networks involved in processing these relationships we used functional magnetic resonance imaging to compare brain activation during processing of behavior-situation and object-situation relationships. Each session examined two aspects of situational relationship processing: monitoring of the situational relationship and responses to irrelevant relationships. Monitoring was analyzed by comparing cortical activation during a situational relevance judgment task with that during a physical appropriateness judgment task. Responses were analyzed by comparing neural responses to situationally irrelevant and situationally relevant components. The left medial frontal cortex, fusiform gyrus, inferior frontal gyrus, calcarine sulcus, right anterior middle temporal gyrus, orbitoinsular junction, and occipito-temporo-parietal junction were commonly activated while monitoring relationships of both types. The right anterior middle temporal gyrus and orbitoinsular junction were considered to have roles in implicit monitoring because they were more deactivated during physical judgment tasks than during the resting state; this deactivation seemed to reflect unconscious situational monitoring in the resting state. Other regions seemed to be linked to explicit conscious monitoring. Responses to irrelevance were linked to separate and category-specific cortical activation in the left medial frontal cortex and frontal pole for behavioral irrelevance and in the left orbitofrontal cortex for irrelevant objects. We demonstrated that the hierarchical structure of processing situational relationships consisted of implicit monitoring, explicit monitoring, and category-specific responses to irrelevance.