Single-trial fluctuations in the EEG signal have been shown to temporally correlate with the fMRI BOLD response and are valuable for modeling trial-to-trial fluctuations in responses. The P1 and N1 components of the visual ERP are sensitive to different attentional modulations, suggesting that different aspects of stimulus processing can be modeled with these ERP parameters. As such, different patterns of BOLD covariation for P1 and N1 informed regressors would be expected; however, current findings are equivocal. We investigate the effects of variations in attention on P1 and N1 informed BOLD activation in a visual oddball task. Simultaneous EEG-fMRI data were recorded from 13 healthy participants during three conditions of a visual oddball task: Passive, Count, and Respond. We show that the P1 and N1 components of the visual ERP can be used in the integration-by-prediction method of EEG-fMRI data integration to highlight brain regions related to target detection and response production. Our data suggest that the P1 component of the ERP reflects changes in sensory encoding of stimulus features and is more informative for the Passive and Count conditions. The N1, on the other hand, was more informative for the Respond condition, suggesting that it can be used to model the processing of stimulus, meaning specifically discriminating one type of stimulus from another, and processes involved in integrating sensory information with response selection. Our results show that an understanding of the underlying electrophysiology is necessary for a thorough interpretation of EEG-informed fMRI analysis.