This study investigated the role of stimulus deviance in determining electrophysiologic and behavioral responses to “novelty.” Stimulus deviance was defined in terms of differences either from the immediately preceding context or from long-term experience. Subjects participated in a visual event-related potential (ERP) experiment, in which they controlled the duration of stimulus viewing with a button press, which served as a measure of exploratory behavior. Each of the three experimental conditions included a frequent repetitive background stimulus and infrequent stimuli that deviated from the background stimulus. In one condition, both background and deviant stimuli were simple, easily recognizable geometric figures. In another condition, both background and deviant stimuli were unusual/unfamiliar figures, and in a third condition, the background stimulus was a highly unusual figure, and the deviant stimuli were simple, geometric shapes. Deviant stimuli elicited larger N2-P3 amplitudes and longer viewing durations than the repetitive background stimulus, even when the deviant stimuli were simple, familiar shapes and the background stimulus was a highly unusual figure. Compared to simple, familiar deviant stimuli, unusual deviant stimuli elicited larger N2-P3 amplitudes and longer viewing times. Within subjects, the deviant stimuli that evoked the largest N2-P3 responses also elicited the longest viewing durations. We conclude that deviance from both immediate context and long-term prior experience contribute to the response to novelty, with the combination generating the largest N2-P3 amplitude and the most sustained attention. The amplitude of the N2-P3 may reflect how much “uncertainty” is evoked by a novel visual stimulus and signal the need for further exploration and cognitive processing.