Timbre characterizes the identity of a sound source. On psychoacoustic grounds, it has been described as a multidimensional perceptual attribute of complex sounds. Using Garner's interference paradigm, we found in a previous behavioral study that three timbral dimensions exhibited interactive processing. These timbral dimensions acoustically corresponded to attack time, spectral centroid, and spectrum fine structure. Here, using event-related potentials (ERPs), we sought neurophysiological correlates of the interactive processing of these dimensions of timbre. ERPs allowed us to dissociate several levels of interaction, at both early perceptual and late stimulus identification stages of processing. The cost of filtering out an irrelevant timbral dimension was accompanied by a late negative-going activity, whereas congruency effects between timbre dimensions were associated with interactions in both early sensory and late processing stages. ERPs also helped to determine the similarities and differences in the interactions displayed by the different pairs of timbre dimensions, revealing in particular variations in the latencies at which temporal and spectral timbre dimensions can interfere with the processing of another spectral timbre dimension.