A socially consequential test of the cognitive penetrability of visual perception is whether merely sharing a group membership with another person influences how you encode their face. Past research has examined this issue by manipulating group membership with techniques from social psychology and then measuring the face-sensitive N170 ERP. However, methodological differences across studies make it difficult to draw conclusions from this literature. In our research, we conducted two large-scale, preregistered ERP studies to address how critical methodological decisions could influence conclusions about top–down effects of group membership on face perception. Specifically, we examined how mere group membership, perceptual markers that signify group membership, number of trials included in the study design, the racial/ethnic identity of face stimuli, and the data analytic approach affect inferences about the N170 response to faces. In Study 1, we found no evidence that mere group membership significantly influenced the N170. However, we found that the background color used to signify group membership modulated the magnitude and latency of the N170. Exploratory analyses also showed effects of stimulus race/ethnicity. In Study 2, we dissociated background color from face encoding by presenting background color before the faces. In this second study, we found no main effect of group membership, background color, or stimulus race/ethnicity. However, we did see an unhypothesized mere group membership effect on trials toward the end of the study. Our results inform debates about social categorization effects on visual perception and show how bottom–up indicators of group membership can bias face encoding.