The time required to find an object of interest in the visual field often increases as a function of the number of items present. This increase or inefficiency was originally interpreted as evidence for the serial allocation of attention to potential target items, but controversy has ensued for decades. We investigated this issue by recording ERPs from humans searching for a target in displays containing several differently colored items. Search inefficiency was ascribed not to serial search but to the time required to selectively process the target once found. Additionally, less time was required for the target to “pop out” from the rest of the display when the color of the target repeated across trials. These findings indicate that task relevance can cause otherwise inconspicuous items to pop out and highlight the need for direct neurophysiological measures when investigating the causes of search inefficiency.