The sensitivity of involuntary attention to top-down modulation was tested using an auditory-visual distraction task and a working memory (WM) load manipulation in subjects performing a simple visual classification task while ignoring contingent auditory stimulation. The sounds were repetitive standard tones (80%) and environmental novel sounds (20%). Distraction caused by the novel sounds was compared across a 1-back WM condition and a no-memory control condition, both involving the comparison of two digits. Event-related brain potentials (ERPs) to the sounds were recorded, and the N1/MMN (mismatch negativity), novelty-P3, and RON components were identified in the novel minus standard difference waveforms. Distraction was reduced in the WM condition, both behaviorally and as indexed by an attenuation of the late phase of the novelty-P3. The transient/change detection mechanism indexed by MMN was not affected by the WM manipulation. Sustained, slow frontal and parietal waveforms related to WM processes were found on the standard ERPs. The present results indicate that distraction caused by irrelevant novel sounds is reduced when a WM component is involved in the task, and that this modulation by WM load takes place at a late stage of the orienting response, all in all confirming that involuntary attention is under the control of top-down mechanisms. Moreover, as these results contradict predictions of the load theory of selective attention and cognitive control, it is suggested that the WM load effects on distraction depend on the nature of the distractor-target relationships.