Absolute pitch (AP) is the rare ability to identify or produce different pitches without using reference tones. At least two sequential processing stages are assumed to contribute to this phenomenon. The first recruits a pitch memory mechanism at an early stage of auditory processing, whereas the second is driven by a later cognitive mechanism (pitch labeling). Several investigations have used active tasks, but it is unclear how these two mechanisms contribute to AP during passive listening. The present work investigated the temporal dynamics of tone processing in AP and non-AP (NAP) participants by using EEG. We applied a passive oddball paradigm with between- and within-tone category manipulations and analyzed the MMN reflecting the early stage of auditory processing and the P3a response reflecting the later cognitive mechanism during the second processing stage. Results did not reveal between-group differences in MMN waveforms. By contrast, the P3a response was specifically associated with AP and sensitive to the processing of different pitch types. Specifically, AP participants exhibited smaller P3a amplitudes, especially in between-tone category conditions, and P3a responses correlated significantly with the age of commencement of musical training, suggesting an influence of early musical exposure on AP. Our results reinforce the current opinion that the representation of pitches at the processing level of the auditory-related cortex is comparable among AP and NAP participants, whereas the later processing stage is critical for AP. Results are interpreted as reflecting cognitive facilitation in AP participants, possibly driven by the availability of multiple codes for tones.