The human visual system can only process a fraction of the information present in a typical visual scene, and selection is historically framed as the outcome of bottom–up and top–down control processes. In this study, we evaluated how a third factor, an individual's selection history, interacts with top–down control mechanisms during visual search. Participants in our task were assigned to one of two groups in which they developed a history of either shape or color selection in one task, while searching for a shape singleton in a second task. A voluntary task selection procedure allowed participants to choose which task they would perform on each trial, thereby maximizing their top–down preparation. We recorded EEG throughout and extracted lateralized ERP components that index target selection (NT) and distractor suppression (PD). Our results showed that selection history continued to guide attention during visual search, even when top–down control mechanisms were maximized with voluntary task selection. For participants with a history of color selection, the NT component elicited by a shape target was attenuated when accompanied by a color distractor, and the distractor itself elicited a larger PD component. In addition, task-switching results revealed that participants in the color group had larger, asymmetric switch costs implying increased competition between task sets. Our results support the notion that selection history is a significant factor in attention guidance, orienting the visual system reflexively to objects that contradict an individual's current goals—even when these goals are intrinsically selected and prepared ahead of time.