It is known that infant faces stimulate visual and anterior brain regions belonging to the mesocortical limbic system (orbito-frontal cortex, anterior cingulate cortex, and nucleus accumbens) as well as the fusiform gyrus during face coding, suggesting a preferential response to baby schema. In the present investigation, faces of infants, children, and adults were presented to 40 male and female right-handed university students with technological objects (and inanimate scenarios to serve as targets) in a randomly mixed fashion. EEG was recorded from 128 scalp sites. In both sexes, the N1 response to infant faces was larger than the response to adult faces; however, the baby-specific N1 response was much larger in women than in men across the left hemisphere. The anterior N2 response to infants was greater than the response to children only in women, whereas the response to children of any age was larger than the response to adults in men. LORETA identified the intracranial sources of N2 response to infants in the left fusiform gyrus (FG), as well as the uncus, cingulate, and orbito-frontal cortices. The FG, the limbic, and especially the orbito-frontal sources were much larger in women than in men. The data suggest a sex difference in the brain response to faces of different ages and in the preferential response to infants, especially with regard to activation of the mesocorticolimbic system.