When a stimulus is associated with a reward, it becomes prioritized, and the allocation of attention to that stimulus increases. For low-level features, such as color, this reward-based allocation of attention can manifest early in time and as a faster and stronger shift of attention to targets with that color, as reflected by the N2pc (a parieto-occipital electrophysiological component peaking at ∼250 msec). It is unknown, however, if reward associations can similarly modulate attentional shifts to complex objects or object categories, or if reward-related modulation of attentional allocation to such stimuli would occur later in time or through a different mechanism. Here, we used magnetoencephalographic recordings in 24 participants to investigate how object categories with a reward association would modulate the shift of attention. On each trial, two colored squares were presented, one in a target color and the other in a distractor color, each with an embedded object. Participants searched for the target-colored square and performed a corner discrimination task. The embedded objects were from either a rewarded or non-rewarded category, and if a rewarded-category object were present within the target-colored square, participants could earn extra money for correct performance. We observed that when the target color contained an object from a rewarded versus a non-rewarded category, the neural shift of attention to the target was faster and of greater magnitude, although the rewarded objects were not relevant for correct task performance. These results suggest that reward associations of complex objects can rapidly modulate attentional allocation to a target.
In any visual scene where multiple objects are present, some objects tend to be particularly effective at capturing and holding attention. Objects are likely to receive attentional priority if they contain bottom–up properties that are of high salience (Theeuwes, 2010; Yantis & Jonides, 1984) or if they contain features that have been associated with reward to an individual, eliciting value-driven capture (Field & Cox, 2008). To an automobile enthusiast, for example, a new sports car may be the most salient object in a parking lot, or an addiction-related object in the environment may be particularly salient to an addict, in both cases capturing attention in a reward-related manner. Objects that contain a rewarding property must first be identified as such (e.g., as a new sports car, a chic item of clothing, a cigarette) to capture attention; however, it is unclear how attentional capture by such complex objects is accomplished and at what stage of processing a reward association with an object might modulate attentional capture. In other words, at what level of neural processing of visual stimuli are reward-associated objects given attentional priority?
One robust neural measure of the allocation of visual attention to a particular stimulus in a scene is the N2pc electrophysiological component. This ERP wave (and its magnetoencephalographic [MEG] counterpart) peaks around 250 msec after stimulus onset and reflects the shift of attention to a target in a visual search display (Hopf et al., 2000; Eimer, 1996; Luck & Hillyard, 1994b). Specifically, this component emerges as a negative polarity wave that is greater contralateral to the target to which attention is shifted, relative to the ipilateral nontarget side (Luck & Hillyard, 1994b), the activity of which originates—aside from a small parietal contribution—mainly from source activity in occipital–temporal cortex (Hopf et al., 2000). Much evidence for the N2pc reflecting a shift in processing has involved target items in an array that are defined by low-level feature singletons such as color or orientation (Woodman & Luck, 1999; Girelli & Luck, 1997; Luck & Hillyard, 1994a), enabling this component to be very successfully and widely used as a marker for the shift or focusing of attention within a visual scene (Woodman & Luck, 2003). More recently, there has been some preliminary evidence to suggest that simple objects can also capture attention in a similar manner to feature singletons. In particular, Wu and colleagues (2013) used letters, numbers, and Chinese characters as stimuli, and they observed that, when participants were searching for an object that fell within a category rather than for a particular item or feature, the N2pc was delayed but still present. This suggests that, when stimuli are more complex and there is not one specific defining feature to search for, the focusing of attention toward the target can occur but is somewhat delayed (and/or is perhaps temporally more variable), implying that objects do not have as much power to capture attention as a color pop-out does.
In addition to it being a marker for attentional orienting, the N2pc is also modulated by reward, with an earlier onset and larger magnitude for reward-associated color targets (Kiss, Driver, & Eimer, 2009). Specifically, in a task in which participants were asked to shift attention to a color pop-out in a display; if that color happened to be associated with a high reward, the onset of the N2pc occurred earlier in time and was larger, suggesting that attention was shifted more quickly and/or more strongly to a reward-associated stimulus. Additionally, MEG and EEG evidence has suggested that a reward-associated color can attract attention without being a target-defining feature when presented as a distractor in a search display (Buschschulte et al., 2014; Hickey, Chelazzi, & Theeuwes, 2010). Thus, reward associations with specific item colors can modulate the orienting and allocation of attention to those items in the environment.
More generally, when a monetary reward is associated with a task, the reward-related aspect has been shown to strongly bias attention toward a spatial location or a feature (see Chelazzi, Perlato, Santandrea, & Della Libera, 2013, for a review). If participants are more likely to receive a monetary reward for detecting targets at a certain location, when targets are subsequently presented (without any prospect of reward) at that location, participants show an attentional bias toward that previously highly rewarded spatial location, suggesting that reward has modulated a spatial priority map of the visual field (Chelazzi et al., 2014) or that attention had been primed to the rewarded location (Hickey, Chelazzi, & Theeuwes, 2014). Other work has suggested that participants can strongly bias their spatial attention in cueing paradigms, if the prospect of a high reward is present (e.g., Small et al., 2005). Many other experiments have examined the effect of reward on attentional processing by binding it to a particular feature (e.g., color). More specifically, when a value is associated with a color, this can increase activity in visual cortices to that color (Serences, 2008), it can speed up and/or enhance the processing of that color (Hopf et al., 2015; Kiss et al., 2009), and it can ramify behaviorally on subsequent trials where the presence of a previously rewarded color serves to distract participants from the current task at hand (Hickey & van Zoest, 2012; Anderson, Laurent, & Yantis, 2011a, 2011b; Hickey et al., 2010). Together, these findings suggest that reward can guide (or misguide) attention to a particular location or feature, giving attentional priority to features or locations that are associated with that reward.
Until very recently, the vast majority of studies on the effects of reward associations have linked reward to a simple feature such as color where these reward associations can easily be made (e.g., Anderson & Yantis, 2011b). Color, however, is a fairly elementary feature, with color pop-outs being known to capture attention effectively, presumably because color is coded comparably early in the visual processing stream (Zeki et al., 1991; Desimone, Schein, Moran, & Ungerleider, 1985). More complex objects that contain many features, on the other hand, have distinguishing characteristics that are processed and extracted at a higher level in the visual stream than color or orientation (Haxby et al., 2001) and, therefore, may not be able to trigger the same rapid, reward-based shift in the capture of attention, given that one would presumably first need to be able to identify the item for attention to be captured by it. Nevertheless, given the behavioral evidence for the capture of attention by rewarding objects (e.g.,Ehrman et al., 2002; Townshend & Duka, 2001), it is possible that enough information about the content of the object could be available early enough (e.g., by 200 msec) to be able to modulate the N2pc. Recently, Hickey, Kaiser, and Peelen (2015) demonstrated in a behavioral study that reward could be associated with a specific type of object (e.g., a person), and if an object from this category (e.g., human beings) were present on a following trial, its presence could distract the participant from the task at hand, causing a slowing down in RTs. Although the stimuli used in this experiment varied, all stimuli could have been identified by the presence of specific features (e.g., a face, limbs), and therefore the categorical association of reward-related effects is still somewhat unknown for categories in which specific features do not clearly define all objects in a category (e.g., furniture).
To determine if and how reward-associated object categories can capture attention, we conducted a reward-association visual search experiment while recording MEG measures of brain activity. We specifically sought to determine whether reward associations for objects might modulate the N2pc in either latency or amplitude, as this would imply that such reward associations were modulating both the rapidity of the attentional shift and the amount of attention allocated to a particular object in the environment at a specific moment in time. To elicit a strong N2pc, we had participants search for a colored square, where this square could have embedded in it an object from a rewarded category (e.g., clothing) or non-rewarded category (e.g., furniture). On the opposite side of the visual field (i.e., the distractor side), there was another colored square that could also either contain an object associated with a reward or an object from a non-rewarded category. Such a design allowed us to distinguish between attention to reward and attention to the processing of the target, as these types of attention are often conflated in studies of reward, although they may be operating as different processes (Maunsell, 2004). We predicted that the presence of a reward-associated item in the target position would result in a more rapid and larger N2pc to that target and that the presence of a reward-associated item in a distractor position would result in a delayed and smaller N2pc to the target on the other side.
Twenty-four healthy, right-handed, paid volunteers participated in this experiment (mean age = 26.5 years, 13 women). Four additional participants were excluded from analysis (one because of poor behavioral performance [<65% correct], two because of excessive eye movements [moving eyes on the majority of the trials], and one because of excessive physiological noise [blinks or other movement artifacts on >35% of trials]). Participants were compensated for their time at a rate of €6 per hour and were additionally rewarded at a rate of €0.025 per correct response (up to €20 total per session; mean total winnings = €17.94, SD = 0.98). All methods and procedures were approved by the ethics board of the Otto-von-Guericke University of Magdeburg.
Each trial began with the presentation of two colored squares, each containing an image of an object, while participants fixated a central fixation cross (see Figure 1). The squares were located below the horizontal meridian in the left and right lower quadrants, each subtending 3.3° × 3.3° of visual angle. The most central corner of each the square was 1.72° below fixation and 2.64° to the left or right of fixation. One color of square was always the target (i.e., to be attended) color (e.g., cyan), and the other color was always the distractor (e.g., yellow) for a given participant. Both the target and distractor squares had corners cut from them that could either be diagonal from each other or on the same side of the square. These cuts subtended 0.77° × 0.77° of visual angle and occurred equally and randomly at all corners within each square. The stimuli were projected onto a screen that was placed 100 cm away from the eyes of each participant in the MEG recording chamber.
Four different categories of objects were used across the experiment (tools, clothing, food, and furniture). Each participant was randomly assigned two of the four categories, one of which was randomly assigned to be the rewarded category for that participant. Every object category contained 40 different images of objects within that category, and each image was originally gray scale on a white background before its colorization by the superposition of a colored square.
Because the rewarded object category could be combined with a target, with a distractor, with both a target and a distractor, or could be not present (i.e., both squares containing items from the non-rewarded object category), there were four different primary trial types in the experiment. These types had the stimulus pairs of targR_distNR, targR_distR, targNR_distR, and targNR_distNR (abbreviations: targ = target, dist = distractor, R = reward, and NR = no reward). Each of these respective trial types could have the target presented on the left or right, which could thus give rise to a total of eight different possible stimulus combinations. Trials in which the target contained the rewarded item could earn the participant an additional 0.025 Euros per correct response if the response (i.e., the cut-corner discrimination) was made between 200 and 1000 msec after the onset of the stimulus.
Before the experiment, each participant completed a flicker-fusion task with the two colors (cyan and yellow) that were used in the experiment. This served to equate the luminance of the target and distractor stimuli for each individual, such that the only difference between them was their respective colors. Additionally, participants were shown ahead of time all 40 objects that they would see in their respective rewarded and non-rewarded categories to ensure that they could identify all items as a member of a given category.
After a short practice of the task, participants were presented with 10 blocks of trials, each block approximately 5 min in duration. Each trial began with the presentation of the bilateral stimulus configuration that remained on the screen for 500 msec. Participants were instructed to shift their covert attention to the square having the designated target color while maintaining central fixation. Once they had shifted their attention to the respective color, they needed to determine whether its corner cuts were on a diagonal or on the same side of the square and to respond accordingly via button press within 1 sec. After the stimuli disappeared, the fixation cross remained on the screen for 700 msec, after which feedback was presented. If answered correctly, participants were told either that they were correct and had earned €0.025 (“Correct + 0.025”) for that trial if a rewarded category object was embedded within the target or just that they were correct (but without the +€0.025) if a non-rewarded category object was embedded within the target. For incorrect trials, participants were told that they were incorrect or too slow to respond, and if the trial had been one in which they could have earned a reward, “+0” was added to the feedback. The feedback was presented for 450 msec, following which the fixation cross remained on the screen for an average of 300 msec (jittered 200–400 msec), giving a total SOA of 650–850 from feedback stimulus onset to the start of the next trial. At the end of each 5-min block, participants were shown the total amount they had earned during that block.
Each condition and left/right configuration (e.g., targR(left)_distNR(right)) was presented a total of 196 times across the entire experiment. The order of the trials was randomized, as were the positions of the cut corners for the target and distractor stimuli. Additionally, the attended color and the categories of objects were randomized and counterbalanced across participants. Because the objects used in the current experiment were images of real objects, which varied in their low-level physical properties, the between-subject counterbalancing served to fully equate for these differences. Additionally, as there were 40 different tokens within each object category, this variety helped to negate any particular low-level feature that could be serving as a categorical identifier.
Continuous MEG data were recorded using a 248-channel, BTI Magnes, whole-head, axiel-gradiometer magnetometer system (4-D Neuroimaging Magnes WH 3600, San Diego, CA). The data were sampled at 254 Hz and filtered online at a low-pass filter of DC to 50 Hz. In addition, environmental magnetic noise was cancelled online based on eight remote reference sensors (see Robinson, 1989, for details). For each participant, the head position was spatially registered (3Space Fastrak System, Polhemus, Colchester, VT) relative to the sensor array by digitizing specific skull landmarks (e.g., nasion, preauricular points), as well as five localizer coils that were attached to the head. Throughout recording, the position of the participants as well as their fixation was also monitored through a closed-circuit video camera, and participants were given verbal feedback on their ability to maintain fixation and head position. Although no eye-tracking data could be recorded, the video feed was monitored by two experimenters to ensure that the participants were maintaining fixation. With the exception of the two participants whose data were omitted from the study because they were observed to not be maintaining good fixation, all participants were able to maintain fixation quite well.
Only trials in which a behavioral response was given within 1 sec of stimulus onset were considered for analysis. The accuracy and RT (correct trials only) for each participant were submitted to a 2 × 2 repeated-measures ANOVA with the factor of Target rewarded (yes/no) and Distractor rewarded (yes/no). The statistical results were corrected for nonsphericity using the Greenhouse–Geisser correction method, and the observed main effects were followed up with specific planned comparisons. All results were considered significant at an alpha of 0.05, unless otherwise specified.
Offline, the MEG data were processed with a denoising algorithm that eliminates slow drift in the data. For each participant, trials containing artifacts (e.g., eye blinks, eye movements, excessive physiological noise) were excluded from analysis. The artifacts were rejected based on a peak-to-peak threshold that was adjusted for each individual (mean rejection threshold = 2.5 pT) and then applied blindly by computer algorithm. On average, 12% of trials (with correct responses) were rejected across participants because of such criteria. For each participant, event-related magnetic field (ERMF) averages time-locked to the onset of the targets were obtained for each trial type, using only trials for which a correct response occurred. A baseline period of 100 msec before stimulus onset was used prior to all statistical comparisons and plots.
To account for differences in the head positions of the participants, the sensor array from each individual was computationally repositioned to a reference sensor set. The reference sensor set, representing the most canonical sensor positions with respect to three key anatomical landmarks (left preauricular, right preauricular, and nasion) had been derived from 1500 prior recording sessions. The first step in the repositioning entailed the computation of a single lead field for each participant based on the anatomical data from the Montreal Neurological Institute (MNI) brain. This resulted in a source space representation (based on the minimum norm least squares algorithm) on the MNI surface. Then, the sensor-space data from each participant were then reconstructed using the inverted lead field of the reference sensor set. This brought the data from each participant into a standard (i.e., coregistered) sensor space that was as if it would have been measured with the sensor reference set.
Source localization of the grand-averaged ERMF responses was performed with Curry (Version 6, Compumedics Neuroscan USA Ltd., Charlotte, NC) through the minimum-norm least-squares algorithm (L2 norm). Source estimates were anatomically constrained by the 3-D surface models (Boundary Element Models; Fuchs, Drenckhahn, Wischmann, & Wagner, 1998) of the source compartment (cortical gray-to-white matter border) and the volume conductor compartment (cerebrospinal fluid space) of the MNI brain.
N2pc Extraction and Statistical Analysis
The N2pc was extracted by taking the ERMF to targets that appeared on the left and subtracting the neural response to targets that appeared on the right. Such a subtraction eliminates the sensory-evoked response (as the display always had bilateral stimulus presentation), isolating the attentional shift-related activity (i.e., the N2pc). Although this subtraction is slightly different from an the extraction of the N2pc for EEG data, wherein activity would be examined as a function of contralateral minus ipsilateral activity, collapsed across targets on both sides, the MEG analogue has been well documented as producing very similar effects, but with higher spatial resolution (see Hopf et al., 2000, for initial confirmation; see Boehler, Tsotsos, Schoenfeld, Heinze, & Hopf, 2011; Finker et al., 2010; Hopf, Boelmans, Schoenfeld, Heinze, & Luck, 2002, for further evidence). This subtraction was run independently for each reward-related condition (e.g., targR_distR) for each participant. The specific sensors of interest were chosen by examining the maximal time point of activity for the N2pc collapsed across the conditions (∼275 msec) and finding those sensors that covered the corresponding influx/efflux extrema. This resulted in four sensors (one left efflux, one left influx, one right influx, one right efflux), the data from which were then rectified so that the influx/efflux had the same polarity and then collapsed across left and right sensors for statistical comparisons and display purposes. The mean amplitude at each sensor for each condition was then extracted and statistically compared across conditions (described in Results).
The onset of the N2pc activity for each condition was determined by a running window t test (window width of three time samples = 11.8 msec) on consecutive time samples between 150 and 250 msec poststimulus onset. For each condition (e.g., targR_distNR), the original waveforms (i.e., targR(left)_distNR(right)) were compared with the targR(right)_distNR(left) to determine when in time these conditions (essentially target left minus target right) started to significantly differ. The onset latencies are reported as the first of multiple consecutive time points (3 or more) where these two conditions significantly differed from each other after correcting for multiple comparisons. To correct for multiple comparisons, the participant's statistically tested data were run through an eigenvalue decomposition to determine the amount of autocorrelation in the data. Specifically, the number of independent variance components that were present over time was determined, and this value was used for correction (see Bartsch et al., 2015; Guthrie & Buchwald, 1991). More specifically, this resulted in three components that had an eigenvalue greater than 1, and this number was applied to the p values using the Bonferroni correction method, such that the new alpha level was 0.017. Of note, the onset values were the same if a jackknife procedure (Miller, Patterson, & Ulrich, 1998) was used in conjunction with the running t tests.
As seen in Figure 2, participants were highly accurate across all trial types, with no main effects on error rate of either the Target being rewarded (F < 1) or the Distractor being rewarded (F < 1). Additionally, there was no interaction between the Target being rewarded and the Distractor being rewarded on the accuracy data (p > .1). RTs, however, were significantly modulated by the presence of a reward in the distractor, F(1, 23) = 12.35, p = .002, ηp2 = 0.35, with slower RTs present when reward was present in the distractor as compared with when it was absent from the distractor. No significant main effect of the presence/absence of reward in the target on the RTs was observed, nor any interaction between the reward being present/absent in the target and present/absent in the distractor (all ps > .1). Thus, although the presence of a rewarding stimulus did not significantly modulate accuracy, participants were significantly slowed down by its presence when it was associated with the distractor.
Across all conditions, there was a clear electrophysiological marker of a shift of attention to the target, as reflected in the ERMF of the N2pc (Figure 3). The presence of the N2pc was confirmed through a two-way repeated-measures ANOVA with the factors Target side (left/right) and Sensor location (influx/efflux), collapsed across reward condition, tested from 175 to 325 msec. This revealed a significant Target × Sensor location interaction, F(1, 23) = 74.92, p < .001, ηp2 = 0.77, confirming the presence of the N2pc. To directly assess the effect of the reward conditions on the extracted N2pc waveforms, a 2 × 2 repeated-measures ANOVA with the factors of Target reward (reward, no reward) and Distractor reward (rewarded, no reward) was run in a time window (175–325 msec) that optimally captured the peak and duration of the N2pc across all conditions. This analysis yielded a significant main effect of Target reward, F(1, 23) = 15.57, p = .001, ηp2 = 0.40, with the amplitude of the N2pc to the target being larger when the target had a rewarded item, and a main effect of Distractor reward, F(1, 23) = 9.13, p = .006, ηp2 = 0.28, with the amplitude of the N2pc to the target being smaller when the distractor had a rewarded item. The relative influence of reward is illustrated in Figure 3C where each of the conditions is compared with when no reward was present at all. As can be seen, having a reward present in both the target and distractor separately influenced the shift of attention, as statistically confirmed in the main effects described above. No significant interaction between these factors was observed (F < 1).
As can be seen in Figure 3, the N2pc appears to start earlier when the target was rewarded and the distractor had no reward associated with it. This was statistically confirmed with the onset of the N2pc for this condition (targR_distNR) occurring at 168 msec, with the other three conditions showing onsets at 180 msec (targR_distR), 180 msec (targNR_distNR), and 183 msec (targNR_distR). Together, these data suggest that when a rewarding stimulus is embedded in the target and not present in the distractor, attention is shifted to the target more rapidly and more strongly.
The source modeling and source waveforms (Figure 4) suggest that the N2pc was originating from similar bilateral temporal-occipital locations, regardless of reward condition. The source activity waveforms (Figure 4B) show a similar pattern to the waveforms obtained at the sensor level (Figure 3), with the N2pc to the target being largest and earliest when a reward is combined with the target but absent in the distractor.
Another way to examine the shift of attention to a reward-associated object is to extract the N2pc reflecting the shift of attention to the reward specifically, regardless of which side the target was on. Thus, we collapsed conditions to extract the N2pc, for when a reward-associated object was on the left versus when it was on the right (leftRew vs. rightRew), the derivation of which is described below and shown in Figure 5A. Specifically, we combined the trials of targR(left)_distNR(right) with the trials of distR(left)_targNR(right), thereby creating a R(left)_NR(right) condition where the activity was collapsed across targets and distractors. We then averaged targNR(left)_distR(right) with distNR(left)_targR(right), such that we ended up with the reward being present on the right side and no reward on the left (NR(left)_R(right)). We then took the difference between these two averages of R(left)_NR(right) minus NR(left)_R(right). Of note, this is analogous to the way the initial N2pc had been extracted, except that instead of the comparison being between when the target was on the left versus right side, it was now between when the reward was on the left versus right side. As seen in Figure 5, this elicited activity similar to the N2pc previously observed for the targets, albeit smaller in magnitude. Statistically, these conditions were significantly different in the same time period and sites as the N2pc above (175–325 msec; F(1, 23) = 19.62, p < .001). The statistical onset of the N2pc to rewarded items, as determined by the moving window t tests, occurred somewhat later than the onset of the N2pc to the target items, beginning at 214 msec (as compared with 168–183 msec, as reported above).
In this study, we examined how a monetary reward that was associated with a category of objects could modulate the latency and magnitude of the shift of attention to a target, as reflected by neural activity measures sensitive to attentional shifts. To this end, participants searched for a colored square (cyan or yellow) and performed a discrimination task concerning the corners of the square. Both the target stimulus and the distractor stimulus had embedded objects, and these objects could be from either a rewarded or non-rewarded category. We observed that when an object of a rewarded category was present within the target stimulus the shift of attention to the color target, as reflected in the N2pc, occurred with a larger magnitude as compared with when a nonrewarded-category object was present in the target stimulus. Furthermore, in the case of the rewarded object being present in the target and not in the distractor, the shift of attention also occurred more rapidly. Finally, when the reward-associated object was present in the distractor stimulus, the N2pc to the target was smaller as compared with when the reward-associated object was absent from the distractor. Such reward-based attentional capture has been shown for colors (Kiss et al., 2009), but here we demonstrate that complex objects that signal a rewarding value can serve to capture attention in a similar manner, both by enhancing attention to a target and by distracting attention away from the target, depending on the location of the reward-associated object.
Although participants were not instructed to shift their attention to the rewarded item and the presence of a rewarded item was manipulated in such a way that it was equiprobable on the target or distractor side, the shift of attention to the target was still substantially modulated by the presence of this rewarding object. Importantly, the rewarded object was not relevant for correct task performance, and thus, the shift of attention to the target could be examined as a function of where the rewarded object occurred. To enable assessment of this possible reward-related modulation, we used the color target as a “carrier” for the shift of attention, thereby preventing participants from having to explicitly shift their attention to objects of specific category (and thus to the rewarding object). Although it is possible to shift attention to objects, the N2pc is substantially reduced when participants shift attention to a category of objects instead of to a specific item (Wu et al., 2013). Such a decreased N2pc would potentially make modulations of the N2pc by reward much smaller and more difficult to observe. By using color as a carrier for the shift of attention, we were able to elicit a robust N2pc, as has been shown in previous studies (e.g., Hopf, Boelmans, Schoenfeld, Luck, & Heinze, 2004; Woodman & Luck, 1999), and then to evaluate its modulation by object-related reward associations. Furthermore, these findings also confirm reports that attentional selection can be modulated by reward, even when the reward is not relevant for correct task performance (e.g., Buschschulte et al., 2014; Libera & Chelazzi, 2006, 2009).
The rapidity with which attention was shifted to the rewarded object (when it was present in the target and absent on the side of the distractor) was somewhat surprising, as it sped up the N2pc's onset to a relatively early latency (∼166 msec). The presence of only two objects in the display as compared with arrays used in more complex visual search tasks likely aided in the rapidity with which attention could be shifted to the target; nevertheless, the presence of a rewarded item in the target side alone accelerated the neural reflection of this attentional shift by approximately 15 msec. That is, by the time of this accelerated N2pc onset, it would seem that the objects must have been identified as members of a particular category (the designated reward-associated one) in order for attention to be shifted more rapidly and strongly to that direction. This task was made more difficult by the fact that there were many different objects within a given category, so participants could not just attend for a specific token of “a chair” or “a shirt,” and therefore, they could not make a specific a priori sensory template for which to attend. Given that previous work has only found a weak N2pc to object categories (Wu et al., 2013), it was not entirely clear that sufficient categorical information of an object could be extracted quickly enough to modulate the shift of attention to a color. Nevertheless, object-category dissociations have been reported to occur neurally as early as by ∼100 msec, (Liu, Agam, Madsen, & Kreiman, 2009) or shortly thereafter (Kirchner & Thorpe, 2006; VanRullen & Thorpe, 2001; Thorpe, Fize, & Marlot, 1996), suggesting that there could be enough time to distinguish between categories by the onset of the N2pc. (However, see work on the N170 as a more robust marker of object category distinction that occurs later in time [e.g., Bentin, Allison, Puce, Perez, & McCarthy, 1996] and work by Evans and Treisman , suggesting that rapid identification of objects is driven by differences in low-level features).
An alternate interpretation of these results would be that the N2pc does not reflect the spatial shift of attention here, but rather the processing of specific features before the actual target selection (Woodman, Arita, & Luck, 2009; Kiss, Van Velzen, & Eimer, 2008). In the study by Kiss and colleagues (2008), when participants had already shifted attention to the side on which the target could appear, an N2pc to this target was still present. If this effect were present in the current data, then one could imagine that attention had been already shifted to the color target, and the N2pc effect could have been the result of the processing of the reward-related object once attention were already present. That is, having the reward present there might both boost the representation of the target while also potentially increasing interference by drawing attention away from the corner task. Whether this was a more rapid attentional shift versus a boost in processing resources once attention was already present is not determinable from the current data; nevertheless, the reward-associated objects received rapid, enhanced processing, demonstrating the ability of object categories to capture attention.
Not only was attention captured by the presence of a reward-associated object in the target stimulus, it was also captured by the presence of a reward-associated object in the distractor. The capture of attention by distracting stimuli can be reflected in the N2pc, as has been demonstrated for non-reward-related contexts (Hickey, McDonald, & Theeuwes, 2006), and the present data help to confirm this capture of attention by the distractor with a rewarded object embedded. It could be the case, however, that, although attention may have initially been captured by the distractor, such attentional capture may have also been later suppressed (Sawaki & Luck, 2010). That is, it is possible that having the reward associated with the distractor may have induced some PD activity, a neural positivity that has been associated with such distractor suppression (Hickey, Di Lollo, & McDonald, 2009), rather than just a shift-related N2pc. The present data cannot distinguish between these two possibilities, although this could be one of the factors that contributed to the results.
This attentional capture by a distractor of the rewarded object category in this study also supports of previous studies that have found costs when a color that has been associated with a reward is associated with a non-target item in a display. Specifically, Hickey and colleagues (2010) observed an N2pc to a distractor stimulus that contained a color that had been associated with a reward. Moreover, several studies have shown that when a reward has been associated with a color, if that color is subsequently present in a display, it will still capture attention even though it is no longer associated with a reward, and such capture can lead to behavioral costs such as slowed RTs (e.g., Chelazzi et al., 2013; Anderson et al., 2011a, 2011b). More recent work has demonstrated that when a reward is associated with a complex object (e.g., a person or car or tree), there are behavioral costs if such an object is present on a subsequent trial (Hickey et al., 2015). Here, we expand such findings with the use of highly diverse object categories, where a single token representation would not help with the rapid classification of an object as a categorical member. Our findings therefore expand upon how reward-related capture of attention can occur in visual search with complex, realistic items that are physically quite variable.
The present findings provide clear neural evidence that reward-associated objects, when present in a target, can speed up and enhance the attentional selection of that target. It is of note, however, that behaviorally we did not observe faster RTs or increased accuracy under these circumstances, indicating that this rapid initial shift of attention toward the rewarded item did not result in a net speed-up of the full cascade of processing that led ultimately to a faster behavioral response. However, there are other facets in the processing further downstream that occur between the attentional selection reflected by the N2pc and the ultimate behavioral RT (e.g., identification/discrimination and response preparation/execution). Accordingly, the more rapid presence of attention on the item would not necessarily mean that there would be a lock-step correspondence in the RT several hundred milliseconds later, indicating that these later facets of the processing required to perform this task were more complex and somewhat independent from the initial shift/focusing of attention. The lack of an effect on the RT, however, does not negate the neural data, which show a robust N2pc to the target containing the reward-associated item. Whether the N2pc reflects the shift of attention to the target or target-specific processing once attention was there, the fact that this component was both larger and earlier in the present data suggests that attention (i.e., the allocation of additional neural resources) was present at the reward-associated item earlier and more robustly.
The behavioral results may seem somewhat surprising given that other tasks in which the reward representation (e.g., color) and the task at hand (e.g., an orientation discrimination) were decoupled have demonstrated that the presence of the reward produced a faster attentional shift to that object and tended to ramify to a more rapid behavioral discrimination of the target (e.g., Kiss et al., 2009). An important distinction between these findings and the current ones, however, is the nature of the stimuli used. First, in the case of the experiment by Kiss and colleagues (2009), the rewarded item was always coupled with the target, which was not always the case in the current study. Furthermore, when reward is associated with a color as it was in the paradigm of Kiss and colleagues (2009), this information can be efficiently processed, leaving resources over to simultaneously do the discrimination task at that location. Here, as the reward was associated with a complex object, discrimination of the smaller details of various features that comprise that object may take up more resources to process. Moreover, the rewarded items here were always located in the center of each image square, whereas the target discrimination processing was directed toward the outer edges of the squares. Thus, the pull of attention inward toward that rewarded item in the square may have tended to delay the RTs relative to when there was no reward present. It is conceivable that the reward-associated objects acted in such a manner as to shift attention more rapidly or strongly toward the target color, in a more broad manner, and after attention was shifted toward the target item, it was relatively more focused on the object itself rather than on the outer edges of the task, which may have tended to slow up the ultimate RT somewhat. Such a notion would be consistent with the findings of Buschschulte et al. (2014), who also had slower RTs on rewarded relative to unrewarded trials. In their design, as here, the rewarded portion of the display could serve as a distractor to capture attention away from the location of the task at hand. Of note, the RTs were slower here when the rewarded item was present in the distractor, suggesting that in this case the rewarded object did pull attention away from the task at hand, which is in line with previous findings (e.g., Hickey et al., 2010).
The shift of attention has been shown to go from a more broad to a more narrow focus in a visual search task (Hopf et al., 2006). In our current data, there may be a slight hint of two separate processes of attentional focusing as it appears that the N2pc was composed of two peaks (an earlier smaller one and a later stronger and broader one; see Figure 3). The early peak of the N2pc may reflect the broad rapid reward-mediated shift of attention to the target color, where as the later peak may reflect the focus on the rewarded object itself, within the target, rather than on the discrimination task at hand. Consequently, the presence of reward in the target may have both attracted attention initially to the correct location and then later attracted attention further, hurting task performance by taking attention away from the corner task. Such a capture of attention by rewarded items in a way that can hurt performance has been observed in studies of stimulus conflict processing, where the association of reward with an irrelevant and conflicting part of a stimulus can lead to slower RTs to the target (Krebs, Boehler, Egner, & Woldorff, 2011;Krebs, Boehler, & Woldorff, 2010). Thus, our results support the idea that reward can not only initially capture attention to an appropriate target location but can also subsequently distract attention from the task at hand if it is not directly relevant for the task.
The results of the source modeling suggested that, in all cases, the shift of attention indexed by the N2pc was arising from the inferior occipital–temporal cortex. This is similar to that which has been observed in other MEG studies (Boehler et al., 2011; Hopf et al., 2000). This source appeared to be weakest under the condition where the rewarded object was embedded in the distractor position and the strongest when it was in the target. Furthermore, the sources of the shift of attention to the rewarded item alone (when collapsed across target and distractor processing; Figure 4) had a similar source distribution as the N2pcs elicited to the target stimuli, suggesting that reward was operating to capture attention in a way similar to the shift of attention to a target, albeit at a smaller magnitude.
In summary, we provide evidence that objects with non-overlapping simple sensory features that are members of a reward-related category can modulate the shift of attention to a color target, as reflected by the earlier latency and enhanced magnitude of the corresponding attention-shifting-sensitive N2pc. These results thus expand upon previous findings showing that reward based on color can robustly capture attention (e.g., Kiss et al., 2009) by showing that reward that is associated with a category of complex stimulus objects can also enhance their processing priority, whether associated with a target or a distractor. Although it is unclear as to how equivalent the salience of a monetary reward is compared with the salience of a primary reward (e.g., water to a thirsty person) or an addiction-related reward (e.g., wine to an alcoholic), nor is it clear how this effect of reward could be parametrically modulated through a high-reward/low-reward manipulation, this work has important implications for the efficacy and manner by which these types of complex object-based stimuli may capture attention. We suggest that reward associations that go beyond just simple low-level features can strongly guide attention in a rapid, automatic manner.
The authors thank Steffi Bachmann and Stefan Knape for technical support and assistance with data acquisition and Anja Rautzenberg and Joseph Harris for helpful input in pilot versions of this experiment. This work was supported by DFG SFB 779 TPA1.
Reprint requests should be sent to Sarah E. Donohue, Department of Neurology, Leibniz Institute for Neurobiology, Otto-von-Guericke-University Magdeburg, Leipziger Strasse 44, 39120 Magdeburg, Germany, or via e-mail: firstname.lastname@example.org.