Where and how does the brain discriminate familiar and unfamiliar faces? This question has not been answered yet by neuroimaging studies partly because different tasks were performed on familiar and unfamiliar faces, or because familiar faces were associated with semantic and lexical information. Here eight subjects were trained during 3 days with a set of 30 faces. The familiarized faces were morphed with unfamiliar faces. Presented with continua of unfamiliar and familiar faces in a pilot experiment, a group of eight subjects presented a categorical perception of face familiarity: there was a sharp boundary in percentage of familiarity decisions between 40% and 60% faces. In the main experiment, subjects were scanned (PET) on the fourth day (after 3 days of training) in six conditions, all requiring a sex classification task. Completely novel faces (0%) were presented in Condition 1 and familiar faces (100%) in Condition 6, while faces of steps of 20% in the continuum of familiarity were presented in Conditions 2 to 5 (20% to 80%). A principal component analysis (PCA) indicated that most variations in neural responses were related to the dissociation between faces perceived as familiar (60% to 100%) and faces perceived as unfamiliar (0 to 40%). Subtraction analyses did not disclose any increase of activation for faces perceived as familiar while there were large relative increases for faces perceived as unfamiliar in several regions of the right occipito-temporal visual pathway. These changes were all categorical and were observed mainly in the right middle occipital gyrus, the right posterior fusiform gyrus, and the right inferotemporal cortex. These results show that (1) the discrimination between familiar and unfamiliar faces is related to relative increases in the right ventral pathway to unfamiliar/novel faces; (2) familiar and unfamiliar faces are discriminated in an all-or-none fashion rather than proportionally to their resemblance to stored representations; and (3) categorical perception of faces is associated with abrupt changes of brain activity in the regions that discriminate the two extremes of the multidimensional continuum.