Although impaired visual search is a core deficit of patients with spatial neglect, current evidence is not conclusive about the mechanisms underlying this failure. We present evidence from 14 neglect patients searching for a target defined by two perceptual features that visual search is mediated by mechanisms of attentional competition. Participants were tested in three search conditions with constant target and distracter positions: Distracters did not share any feature with the target; distracters shared one feature with the target; two distracters shared one feature and one distracter shared the other feature with the target (mixed condition). Whereas search performance of healthy participants was comparable across conditions, neglect patients had a significant contralesional slowing in the mixed condition compared with the other two conditions. A detailed lesion analysis revealed that involvement of the parietal lobe did not predict the degree of distractibility in visual search. In contrast, neglect patients with high distractibility had more frequent damage to the inferior temporal lobe, suggesting a preliminary role of this region for competitive attentional processes involved in visual search of spatial neglect patients.