In the lateralized change detection task, two item arrays are presented, one on each side of the display. Participants have to remember the items in the relevant hemifield and ignore the items in the irrelevant hemifield. A difference wave between contralateral and ipsilateral slow potentials with respect to the relevant items, the contralateral delay activity, can be calculated. As its amplitude varies with the number of items held in working memory (WM) and reaches its asymptote with WM capacity, it is considered a pure neural correlate of visual WM load. However, in addition to this contralateral delay activity, load-dependent activity has also been observed over the hemisphere ipsilateral to the relevant hemifield, suggesting that the ipsilateral hemisphere is also involved in memory-related processes. This ipsilateral activity might either reflect a bilateral processing of relevant or else a lateralized processing of irrelevant, to-be-filtered-out items. As in the lateralized change detection task, the number of items on both sides of the display is typically identical, it was not possible to decide between these alternatives yet. To disentangle the influence of relevant and irrelevant items, we orthogonally varied the number of both types of items. Processing of relevant items caused purely contralateral load-dependent activity. Ipsilateral slow potentials were influenced by the number of irrelevant items only if visual WM load was low, but not if it was high. This suggests that whether irrelevant items are processed or filtered out depends on visual WM load.