Faces can be represented at a variety of different subordinate levels (e.g., race) that can become “privileged” for visual recognition in perceivers and is reflected as patterns of biases (e.g., own-race bias). The mechanisms encoding privileged status are likely varied, making it difficult to predict how neural systems represent subordinate-level biases in face processing. Here, we investigate the neural basis of subordinate-level representations of human faces in the ventral visual pathway, by leveraging recent behavioral findings indicating the privileged nature of peer faces in identity recognition for adolescents and emerging adults (i.e., ages 18–25 years). We tested 166 emerging adults in a face recognition paradigm and a subset of 31 of these participants in two fMRI task paradigms. We showed that emerging adults exhibit a peer bias in face recognition behavior, which indicates a privileged status for a subordinate-level category of faces that is not predicted based on experience alone. This privileged status of peer faces is supported by multiple neural mechanisms within the ventral visual pathway, including enhanced neural magnitude and neural size in the neural size in the fusiform area (FFA1), which is a critical part of the face-processing network that fundamentally supports the representations of subordinate-level categories of faces. These findings demonstrate organizational principles that the human ventral visual pathway uses to privilege relevant social information in face representations, which is essential for navigating human social interactions. It will be important to understand whether similar mechanisms support representations of other subordinate-level categories like race and gender.