Extensive studies have focused on selection mechanisms during visual search. One important influence on these mechanisms is the perceptual characteristics of the stimuli. We investigated the impact of perceptual similarity between targets and nontargets (T-N similarity) in a visual search task using EEG. Participants searched for a predefined target letter among five nontargets. The T-N similarity was manipulated with three levels: high, middle, and low. We tested for the influences of T-N similarity on an ERP (e.g., N2pc) and alpha oscillations. We observed a significant N2pc effect across all levels of similarity. The N2pc amplitude was reduced and occurred later for high similarity relative to low and middle similarities. We also showed that the N2pc amplitude was inversely correlated with the RTs across all similarities. Importantly, we found a significant alpha phase adjustment about the same time as the N2pc for high similarity; by contrast, no such effect was observed for middle and low similarities. Finally, we showed a positive correlation between the phase-locking value and the N2pc—the stronger the alpha phase-locking value, the larger the N2pc, when the T-N similarity was high. In conclusion, our results provide novel evidence for multiple competitive mechanisms during visual search.
Goal-directed selection of visual items from the external environment relies upon our ability to identify stimulus properties that are most relevant to the task in hand, often while simultaneously ignoring behaviorally irrelevant distractions. Many studies have focused on selective mechanisms during visual search, isolating the cognitive and neural processes that enable us to identify the presence of a specific target among irrelevant nontargets or distractors (Eimer, 2014; Chelazzi, 1999; Wolfe, 1994; Duncan & Humphreys, 1989; Treisman & Gelade, 1980). The characteristics of visual stimuli can heavily influence the extent to which attention can be effectively allocated (Itti & Koch, 2001; Desimone & Duncan, 1995). The efficiency of searching for predefined targets may depend on the properties of surrounding objects. When distractors are more similar to one another, it is easier to identify the target. In addition, the perceptual similarity between targets and nontargets (T-N similarity) can also affect the efficiency of visual search—when targets are more dissimilar from the distractors, it is easier to find (Duncan & Humphreys, 1989). Here, we investigated how neural activity is modulated as we search for targets, using EEG, and in particular whether this is sensitive to T-N similarity.
There are many theoretical accounts of visual search (Eimer, 2014; Wolfe, 1994; Duncan & Humphreys, 1989; Treisman & Gormican, 1988), and most posit a strong role for the visual properties of the stimuli in the array or scene. According to one popular account (Kastner & Ungerleider, 2000, 2001; Desimone & Duncan, 1995), when multiple items are simultaneously presented in a visual display, they compete for neural representation through mutually suppressive interactions. According to this account, the processing capacity for visual information is limited, and this creates a bottleneck for selection. This competitive process can be biased via top–down mechanisms to ensure that relevant information is represented (Buschman & Kastner, 2015; Reynolds & Chelazzi, 2004; Kastner & Ungerleider, 2000; Desimone & Duncan, 1995; Bundesen, 1990). Evidence in favor of this account comes from many sources. Neurophysiological evidence in macaques has revealed that neuronal responses are reduced when two visual stimuli are simultaneously presented in the same receptive field of neurons in visual extrastriate areas (Reynolds & Chelazzi, 2004; Moran & Desimone, 1985). Studies in humans using fMRI also showed a reduction in the neural response in the visual cortex to stimuli under conditions of competition (Scalf & Beck, 2010; Beck & Kastner, 2009). These findings suggest that attention biases competitive interaction to the attended target in visual areas (Reynolds, Chelazzi, & Desimone, 1999; Luck, Chelazzi, Hillyard, & Desimone, 1997).
The time course of competitive bias toward target representations in visual search is relatively unclear. To address this, we exploited an ERP, namely, the N2pc, and oscillatory activity in the alpha band (9–13 Hz) as indices of spatiotopically organized selection processes during visual search. The N2pc is typically observed between 200 and 400 msec after the onset of the search array, with more negative potentials over electrode sites contralateral to the attended hemifield (Eimer & Grubert, 2014; Töllner, Gramann, Müller, Kiss, & Eimer, 2008; Hopf et al., 2000; Luck & Hillyard, 1994). The N2pc is closely related to the selection of targets based upon template matching (Eimer & Grubert, 2014; Brignani, Lepsien, & Nobre, 2010; Kuo, Rao, Lepsien, & Nobre, 2009; Eimer, 1996). Other lateralized electrophysiological effects have also been linked with visual search. For example, oscillatory power in the alpha band at posterior brain regions shows decreased power contralateral to the attended hemifield and increased power contralateral to the unattended hemifield (Payne, Guillory, & Sekuler, 2013; Klimesch, 2012; Händel, Haarmeier, & Jensen, 2011; Rihs, Michel, & Thut, 2009; Sauseng et al., 2009; Kelly, Lalor, Reilly, & Foxe, 2006; Thut, Nietzel, Brandt, & Pascual-Leone, 2006; Worden, Foxe, Wang, & Simpson, 2000). One widely held view is that alpha power lateralization reflects a top–down modulation of neuronal excitability/inhibition that serves to gate incoming sensory information, with selective enhancement of targets and suppression of irrelevant nontargets or distractors (Payne & Sekuler, 2014; Zumer, Scheeringa, Schoffelen, Norris, & Jensen, 2014; Händel et al., 2011). Recent EEG studies using forward or inverted encoding models further suggest that alpha power topographies could covary with the exact location of an attended item rather than just the hemifield (Foster, Sutterer, Serences, Vogel, & Awh, 2017; Samaha, Sprague, & Postle, 2016). Similarly, the phase of alpha oscillation that reflects rhythmic changes of fluctuating states of neuronal excitability or inhibition has also been shown to modulate visual perception and awareness (Romei, Gross, & Thut, 2012; Dugué, Marque, & VanRullen, 2011; Busch, Dubois, & VanRullen, 2009; Mathewson, Gratton, Fabiani, Beck, & Ro, 2009). The adjustment of alpha phase may indicate the timing of the modulation by the top–down mechanisms in the visual cortex on a fast timescale (Jensen, Bonnefond, & VanRullen, 2012; Klimesch, 2012; Haegens, Nácher, Luna, Romo, & Jensen, 2011; Klimesch, Sauseng, & Hanslmayr, 2007).
Here, we investigated the electrophysiological correlates of T-N competitive interactions during visual search. T-N similarity was manipulated with three levels: high, middle, and low. The more similar the nontargets are to the target, the more they should compete with it for representation. As a result, the degree of top–down bias needed in order to select the target should increase. If either the N2pc or alpha oscillation reflects this process, then it should be influenced by T-N similarity. Moreover, we conducted a model-based analysis (van den Berg, Appelbaum, Clark, Lorist, & Woldorff, 2016; Myers, Stokes, Walther, & Nobre, 2014) to explore the relationships between N2pc and search performance on a trial-by-trial basis. We also tested for the relationship between the N2pc and alpha oscillation for visual search. Differential relationships between the neural markers and between brain activity and behavior could provide strong evidence for the roles of these two mechanisms (e.g., N2pc and alpha oscillations) in visual search.
Twenty healthy volunteers (12 women, age range = 20–27 years, mean age = 22.25) participated in this study. All participants were right-handed according to the Edinburgh Handedness Inventory (Oldfield, 1971). The participants had normal or corrected-to-normal visual acuity, provided informed written consent before the experiment, and were financially reimbursed for their time (500 NTD, about 16 USD). All experimental methods and procedures were approved by the research ethics Office of the National Taiwan University.
Stimuli were presented on a 17-in. cathode ray tube monitor with a refresh rate of 60 Hz using the E-Prime software (Psychology Software Tools, Inc.). We selected 12 English letters based on the similarity measures of letter identification from previous findings (Mueller & Weidemann, 2012; Townsend, 1971). The target letters were X and N (Lavie & Cox, 1997). The similar nontarget letters with linear characteristics included H, K, M, V, and Y; the dissimilar nontarget letters with curved characteristics included C, G, O, Q, and D. Each letter stimulus subtended a visual angle of approximately 0.42° (height) × 0.42° (width) and was presented in white, capital Courier New font. The target and nontarget letters were positioned in six possible peripheral locations of an invisible circle that subtended approximately 1.87° in radius with equal space between adjacent letters. All of the stimuli were presented against a black background.
The experimental design followed a 3 (T-N similarity: high, middle, low) × 2 (target side: left, right) within-subjects factorial design. The five nontarget letters with linear characteristics were presented for high-similarity trials, and the five nontarget letters with curve-shaped characteristics were presented for low-similarity trials. Three curve-shaped nontarget letters and two linear nontarget letters were presented for middle-similarity trials. The relative locations among the target letter and linear and curve-shaped nontarget letters were fully counterbalanced. The task procedure is illustrated in Figure 1.
Each trial began with the onset of a centrally displayed fixation for 500 msec, which signaled the onset of the trial. After that, a search array consisting of six stimulus letters was presented for 100 msec and followed by a maximal 1100 msec for response. Participants were instructed to search for a predefined target letter (X or N) and ignore the nontarget letters. Half of the participants were instructed to press the left mouse button for the target “X” using their left thumb and the right mouse button for the target “N” using their right thumb, whereas the other half were instructed oppositely. The interval between trials varied randomly between 800, 1000, and 1200 msec.
Participants were comfortably seated in a dimly illuminated room, facing a cathode ray tube monitor placed 57 cm in front of them. Before the formal experiment, the participants were given written, as well as verbal, instructions about the task requirements. Participants first completed a practice block of 36 trials to ensure that they could perform the task as instructed. They were encouraged to rest between blocks that they could self-initiate. In the formal experiment, all trial types were equiprobable and randomized within 30 blocks of 36 trials, yielding 1080 trials in total (360 trials for each high-, middle-, and low-similarity condition). Participants were instructed to maintain fixation on a small fixation marker at the center of the screen during the experimental trials and to respond as quickly and accurately as possible. Participants responded using both hands by pressing the mouse buttons. The participants were also asked to minimize blinking and movement of their eyes while performing each trial throughout the experiment. The total experimental time for each participant was approximately 60 min.
The behavioral measures, including accuracy and RTs, were each analyzed by one-way (T-N similarity: high, middle, low) repeated-measures ANOVA. Only correct responses were included in the RT analyses.
The EEG data were recorded continuously using a NuAmp amplifier (Neuroscan, Inc.), with 37 Ag/AgCl electrodes placed on the scalp with an elasticated cap positioned according to the 10–20 international system. The montage included six midline sites (Fz, FCz, Cz, CPz, Pz, and Oz) and 12 sites over each hemisphere (FP1/FP2, F3/F4, F7/F8, FC3/FC4, FT7/FT8, C3/C4, T3/T4, CP3/CP4, TP7/TP8, P3/P4, T5/T6, and O1/O2). Vertical eye movements were recorded by electrodes placed on the supraorbital and infraorbital ridges of the right eye (vertical EOG [VEOG]), and horizontal eye movements were recorded by electrodes placed on the outer canthi of the right and left eyes (horizontal EOG [HEOG]). Additional electrodes served as the ground (AFz, positioned between FPz and Fz) and reference (A1 and A2 mastoid sites, with A2 serving as the active online reference). Electrode impedances were kept below 5 KΩ. Ongoing brain activity at each electrode site was sampled every 1 msec (1000-Hz analogue-to-digital sampling rate). The activity was filtered with a low-pass filter of 300 Hz, and no high-pass filter was used. Digital stimulus presentation codes were sent to the EEG acquisition computer via a parallel port from the stimulus presentation computer, indicating the type and exact time of presentation for each event.
Offline, the EEG data were re-referenced to the algebraic average of the right and left mastoids (A1 and A2). Bipolar EOG signals were derived by computing the difference between the voltages at electrodes placed to the side of each eye (HEOG) and above and below the left eye (VEOG). The continuous EEG data were then segmented into epochs starting 500 msec before and ending 1000 msec after the onset of the stimuli. The EEG epochs were baseline-corrected, with a 100-msec prestimulus period. Epochs containing excessive noise or drift (±100 μV) at any electrode were excluded. Epochs with ocular artifacts (blinks or saccades) were also rejected in two steps. Blinks were identified as large deflections (±60 μV) in the HEOG or VEOG electrodes. Visual inspection was then carried out to confirm the appropriate removal of artifacts and identify residual saccades or eye movements (e.g., boxcar-shaped voltage deflection) in individual HEOG traces (±10–20 μV). Trials with incorrect behavioral responses were also discarded from further analyses.
Epochs were then averaged according to T-N similarity and target side. For each condition, ERPs were derived from the onset of the search arrays. ERPs from trials containing targets located on the right and from trials containing targets located on the left side were combined by an averaging procedure that preserved the electrode location relative to the target side (contralateral or ipsilateral side to the target). To maintain an acceptable signal-to-noise ratio, a lower limit of 100 artifact-free trials per subject per condition was set.
We first tested for the occurrence of the N2pc potential across three types of T-N similarity. Statistical analyses for the N2pc were performed using the mean voltage value (mean amplitude) over successive time bins across different scalp regions. The mean amplitudes of the N2pc were measured between 200 and 320 msec at the occipital-temporal electrodes (e.g., the averaged amplitude of T5/T6 and TP7/TP8) contralateral and ipsilateral to the side of the target (e.g., Eimer, 1996; Luck & Hillyard, 1994). To test the time course of the influences of T-N similarity on visual search, we divided 200–320 msec into three time windows: 200–240 msec, 240–280 msec, and 280–320 msec. A three-way repeated-measures ANOVA was computed of the mean amplitudes of the N2pc, testing the effects of T-N similarity (high, middle, low), Time window (200–240 msec, 240–280 msec, 280–320 msec), and Target side (contralateral and ipsilateral to target). Only effects including the factor of Target side, reflecting differential activity observed in ipsilateral versus contralateral electrodes, were of interest. We also confirmed the N2pc effect (ipsilateral side relative to contralateral side to target) for each similarity condition for each time window using paired t tests. The Greenhouse–Geisser epsilon correction for nonsphericity was applied to all ERP analyses where appropriate (Jennings & Wood, 1976), and only corrected probability values and degrees of freedom are reported.
Relationship between N2pc and RTs
We conducted a model-based analysis (Myers et al., 2014) to test for the relationship between the N2pc and the RTs at two levels. At the individual level, we calculated the N2pc by subtracting the amplitude at the contralateral side from the ipsilateral side relative to the target location for each trial for each of three time windows (200–240 msec, 240–280 msec, 280–320 msec), respectively. We fitted a general linear model (GLM) to the z-transformed N2pc amplitude using the z-transformed RT as a regressor at each of time point for each electrode of interest (e.g., T5/T6 and TP7/TP8) for each N2pc time window on a trial-by-trial basis. The GLM was fitted for each of the T-N similarity conditions separately and then averaged regression weights (beta estimates) across conditions and the electrodes of interest. This was repeated for each participant's ERP data. Individual data were then entered into the group-level analysis. We used one-sample t tests to examine a significant relationship between N2pc and RTs (p < .05, two-tailed). In essence, this tests whether the variability in RT is significantly predicted by the trial-wise variability in the magnitude of the N2pc effect.
EEG Time–Frequency Analysis
The statistical analyses were computed using a cluster-based nonparametric permutation approach (Maris & Oostenveld, 2007) across participants for each T-N similarity condition. This analytical approach controlled for Type I errors by calculating the size of any significant clusters with consecutive t tests that were significant (p < .05) across neighboring electrodes, time points, or both, with averaging across the alpha band (9–13 Hz). For the statistical analysis of lateralized power and PLVs in the alpha range for each T-N similarity condition, we compared the differences between the ipsilateral versus contralateral to the target hemifield after the offset of the search array. We then tested the statistical significance using dependent sample t tests by calculating Monte Carlo p values on 1000 random partitions, in which two conditions were shuffled to create the null distribution of the clustered test statistics that would be achieved by chance. We estimated the corrected p value according to the proportion of the null distribution exceeding the observed cluster-level t statistics. If its size was unlikely to have occurred by chance (p < .05, two-tailed), the cluster was treated as significant.
Correlation between Lateralized Alpha Power/PLVs and N2pc
We also tested for the relationship between the lateralized alpha power/PLVs and the N2pc in the model-based analysis at the group level. For each time window of N2pc and for each of the T-N similarity conditions, we fitted the GLM to the z-transformed lateralized alpha power and PLVs (averaged 9–13 Hz and electrodes of interest), respectively, using the z-transformed mean N2pc (averaged electrodes of interest) as a regressor. Three N2pc effects (from 200 to 240 msec, from 240 to 280 msec, and from 280 to 320 msec) were used to fit each GLM. We then evaluated the strength of correlation between lateralized alpha power and N2pc and between lateralized alpha PLV and N2pc on the basis of a jackknife approach (Miller, Patterson, & Ulrich, 1998). We iteratively removed one participant from all the participants and thus resulted in 20 data sets. This leave-one-participant-out approach allows us to obtain a jackknife estimate of the reliability of the correlation and the estimate of the standard error allows us to compare the beta estimates against zero (i.e., the null hypothesis of no correlation) using one-sample t tests and the Type I error were corrected using Bonferroni correction for three time windows (p < .0167, two-tailed).
Analysis of Temporal Relationship between Lateralized Alpha PLV and N2pc
Finally, we explored the temporal relationship between the time courses of lateralized alpha PLV and N2pc for high similarity using a cross-correlation analysis (van Ede, Chekroud, Stokes, & Nobre, 2019). We calculated the cross-correlational coefficients between these two time courses from 200 to 320 msec using the xcov function in MATLAB for each participant. We set the range of time lag from −40 to 40 msec, shifting PLV time course in a 4-msec step on the basis of the N2pc time course. The cross-correlation coefficient varies from −1 to 1, where 1 (−1) indicates positive (negative) correlation between two signals and 0 indicates no correlation. The maximal coefficient at negative or positive time lag indicates the temporal difference in correlation between the time courses of lateralized alpha PLV and N2pc (negative time lag: lateralized alpha PLV occurs before the N2pc; positive time lag: the N2pc occurs before the lateralized alpha PLV). On the other hand, the maximal coefficient at zero lag indicates no temporal difference between the two time courses. The cross-correlation coefficients were transformed to z values using Fisher's z transform for each participant. We used one-sample t tests (p < .05, one-tailed) to compare the Fisher's z scores against zero (i.e., the null hypothesis of no correlation) for each time lag. If the lateralized alpha PLV occurs preceding the N2pc for high similarity, we would expect maximal correlation between the two time courses at negative time lag.
The behavioral results are illustrated in Figure 1. The accuracy data showed a significant main effect of T-N similarity, F(2, 38) = 181.496, p < .001, ηp2 = .905. Tukey's honestly significant difference post hoc comparisons indicated that the low-similarity condition (M = 0.88, SE = 0.023) was more accurate than the middle- (M = 0.76, SE = 0.023) and high-similarity (M = 0.65, SE = 0.02) conditions, and the middle-similarity condition was more accurate than the high-similarity condition. The analysis of RT data revealed a significant main effect of the T-N similarity, F(2, 38) = 63.146, p < .001, ηp2 = .769. Post hoc comparisons indicated that the RTs for the low-similarity condition (M = 620.25, SE = 25.206) were faster than those for the middle- (M = 675.69, SE = 27.652) and high-similarity (M = 715.85, SE = 31.51) conditions, and those of the middle-similarity condition were also faster than those of the high-similarity condition. These results demonstrated the influence of T-N similarity on visual search performance—as targets and distractors became more similar, subjects became slower and more error prone in their search.
The ERP results are shown in Figure 2. The ERP results showed a significant main effect of Target side, F(1, 19) = 41.90, p < .001, suggesting an N2pc effect. We also observed significant main effects of Time window, F(1.377, 38) = 5.46, p = .019, significant two-way interactions between the T-N similarity and Time window, F(3.059, 76) = 12.52, p < .001, Target side and Time window, F(1.632, 38) = 27.36, p < .001, T-N similarity and Target side, F(2, 38) = 13.31, p < .001, and a significant three-way interaction among T-N similarity, Target side, and Time window, F(2.729, 76) = 9.79, p < .001. To clarify these interactions and the N2pc effects, we tested the impact of T-N similarity on the N2pc for each time window in the follow-up analyses below.
For the time window of 200–240 msec, the N2pc effects were significant for the middle-, t(19) = 2.47, p = .023, and low-similarity, t(19) = 3.67, p = .002, trials. The difference in N2pc amplitudes among the three types of similarity, however, was not significant during this time window, F(2, 38) = 1.86, p = .17. Next, we observed significant N2pc effects for the high-, t(19) = 2.30, p = .033, middle-, t(19) = 4.72, p < .001, and low-similarity, t(19) = 5.16, p < .001, trials during 240–280 msec and significant N2pc effects for the high-, t(19) = 2.66, p = .015, middle-, t(19) = 4.86, p < .001, and low-similarity, t(19) = 6.69, p < .001, trials during 280–320 msec. Finally, we found a significant effect of the T-N similarity on the N2pc amplitude from 240 to 280 msec, F(2, 38) = 11.56, p < .001, and from 280 to 320 msec, F(2, 38) = 17.86, p < .001. These results showed a reduction in the N2pc amplitude during the high-similarity trials relative to that of the low-similarity trials from 240 to 320 msec and a reduction in the N2pc amplitude during the high-similarity trials relative to the middle-similarity trials from 280 to 320 msec. No significant differences in the N2pc effect between the low- and middle-similarity trials were found across the three time windows.
Relationship between N2pc and RTs
Next, we conducted a trial-by-trial model-based analysis to test for the relationship between the N2pc and the RTs. We showed a significant negative relationship between the N2pc and RTs during the time window of 240–280 msec (p = .004) and 280–320 msec (p < .001). These results showed that N2pc amplitude was inversely correlated with the RTs across all similarity conditions—the bigger the N2pc, the faster the RT on that trial (Figure 2C).
EEG Time–Frequency Results
We then tested whether the T-N similarity can influence lateralized power and PLVs in the alpha band (ipsilateral vs. contralateral side). We identified a significant effect for high T-N similarity trials with increased PLV over the ipsilateral side to the target location and decreased PLV over the contralateral side to the target location, which began at 224 msec and extended to 276 msec (corrected p = .027), over occipital-temporal electrodes (T5/6 and TP7/8). However, no such effect was observed for the middle- and low-similarity trials. We did not observe any significant effect for lateralized alpha power for all three types of similarity. The time–frequency results are illustrated in Figure 3.
Correlation between Lateralized Alpha Power/PLVs and N2pc
We tested for the relationship between lateralized alpha power/PLVs and N2pc for the high T-N similarity trials. We found that the strength of lateralized PLV (ipsilateral vs. contralateral side) was positively correlated with the N2pc amplitude (ipsilateral vs. contralateral side) from 240 to 280 msec, tc(19) = 4.97, p < .001, and from 280 to 320 msec, tc(19) = 4.93, p < .001 (Figure 4A). No other significant effect was observed.
Temporal Relationship between Lateralized Alpha PLV and N2pc
Finally, we calculated cross-correlations between the time courses of lateralized alpha PLV and N2pc for each participant. We found that the strength of lateralized PLV was positively correlated with the N2pc amplitude from −16 to 0 msec time lags of N2pc (ps < .05), but no significant effect was found at other time lags. The cross-correlational results are illustrated in Figure 4B.
The current study aimed to investigate the influences of T-N similarity on the electrophysiological activity during visual search. Three levels of T-N similarity, namely, high, middle, and low, were manipulated. Our ERP results showed a significant N2pc effect across all levels of T-N similarity. Particularly, the N2pc amplitude was reduced and occurred later for high similarity relative to those for low and middle similarities. We also showed a close relationship between N2pc and behavior. This result indicated that the N2pc amplitude was inversely correlated with the RT across all similarity conditions. Importantly, we found a significant lateralization of alpha phase-locked effect about the same time as the N2pc for high T-N similarity, but no such lateralization was observed for middle or low T-N similarities. Finally, we showed a positive correlation between the lateralized PLV and the N2pc—the stronger the PLV in the alpha band, the larger the N2pc, when the T-N similarity was high.
As a large body of ERP evidence has shown that the contralateral N2pc is elicited by the presence of task-relevant attributes during visual search (Eimer, 1996, 2014; Luck & Hillyard, 1994), we found N2pc effects across all levels of T-N similarity. The N2pc reflects a selective process that indicates the spatiotopic layout of a putative target map that results from preferential processing of the relevant representations of the target identity, that is, attentional templates (Grubert & Eimer, 2015; Nako, Smith, & Eimer, 2015). The presence of the N2pc across the three types of T-N similarity suggested the top–down selection of spatiotopic sensory codes based on template matching occurs in all conditions. But this process is variable across trials, and this variability mirrors target detection times—when the N2pc is large, it corresponds to targets being found more quickly. ERP studies that have combined visual search with cueing paradigms have shown that the N2pc is related to the selection of the visual target at specific locations instead of anticipatory attention per se (Woodman, Arita, & Luck, 2009; Kiss, Van Velzen, & Eimer, 2008). When observers search for a predefined target among multiple elements from within a stimulus array, a target template can guide allocation of attention for target selection (Olivers, Peters, Houtkamp, & Roelfsema, 2011; Duncan & Humphreys, 1989).
In accordance with the predictions based on the competition account (Desimone & Duncan, 1995), we showed that search efficiency was influenced by the T-N similarity. The behavioral performance suggested that searching was less efficient when the targets and nontargets were highly similar. The pattern of the ERP (i.e., the N2pc) mirrored the pattern of search performance, suggesting a later and reduced N2pc, as well as slower and less accurate searching, for high-similarity trial relative to that for middle- and low-similarity trials. ERP studies that have manipulated distractor heterogeneity demonstrated reduced N2pc amplitudes when targets were presented in more heterogeneous, random context relative to less heterogeneous, grouped context (Feldmann-Wüstefeld & Schubö, 2015; Feldmann-Wüstefeld, Wykowska, & Schubö, 2013). In line with these findings, the reduction in the N2pc for high-similarity trials in the current study may reflect the competitions of neural representations during the target selection process (Hilimire, Mounts, Parks, & Corballis, 2009)—this competition takes longer to resolve, reflecting a slowed template matching conclusion alongside increased RTs. Similar nontarget stimuli may activate part of the target representations and this in turn causes the competition (cf. Bundesen, 1990). Such competition results in mutual suppression to degrade the preferential processing of target selection and makes the search process less efficient.
Importantly, a novel finding shows that alpha phase adjustment is also sensitive to the competition during visual search. The cortical rhythm in the alpha band can adjust in phase over posterior brain sites only when T-N similarity was high, with increased PLV in the ipsilateral site and decreased PLV in the contralateral site relative to the target location. This modulation of T-N similarity of PLV began approximately 224 msec after the onset of the search array. This effect was not present for the middle- or low-T-N-similarity trials. This finding is consistent with the notion that the alpha oscillation consists of a selective mechanism for enhancing relevant attributes and a suppressive mechanism for keeping irrelevant nontargets or distractors from limited neural representations (Kuo, Li, Lin, Hu, & Yeh, 2017; Payne et al., 2013; Jensen et al., 2012; Klimesch, 2012; Haegens et al., 2011; Rihs et al., 2009; Sauseng et al., 2009; Klimesch et al., 2007; Kelly et al., 2006; Thut et al., 2006; Worden et al., 2000). An adjustment in alpha phase may provide a modulatory mechanism for inhibiting nontarget items or distractors on a fast timescale for target selection. The positive correlation in the lateralized measurement (ipsilateral vs. contralateral side to the target location) between the PLV and N2pc for the high-similarity trial validated this view, suggesting a greater difference in alpha phase adjustment with a larger difference in the N2pc. One possibility is that this early effect in alpha phase reflects the early biasing of sensory processing. Under conditions of maximal competition a stronger bias is needed.
Although the PLV (from 224 msec) seems to occur earlier than the N2pc (from 240 msec) when the targets and nontargets were highly similar, the temporal relationship between these two signals is unclear. To address this issue, we calculated cross-correlational coefficients between the time courses of the lateralized alpha PLV and N2pc and observed the maximal positive correlations at negative time lags from −16 to 0 msec of N2pc. This analysis of time courses reflects that the lateralized alpha PLV could occur preceding or at the same time as the presence of N2pc but not later than the N2pc for high-similarity trials. These results also suggest that searching for a target among competitive nontargets triggers two distinct mechanisms of top–down control: alpha oscillation and N2pc. A recent study using a visual search paradigm provided complementary evidence to highlight this point of view (Bacigalupo & Luck, 2019). In their study, both N2pc and lateralized alpha suppression in power were observed when the target was presented in the lower visual field. These two lateralized effects also exhibited different time courses and responded differently to the effects of visual crowding. In line with their findings, we found that the alpha phase adjustment can also be partly dissociated from the N2pc in time and both N2pc and lateralized alpha PLV responded differently to T-N similarity.
Our results suggest that both top–down and bottom–up characteristics drive the priority in selection during visual search. Goal-relevant target templates hold in working memory bias selection toward the locations of stimuli that share similar features with the templates. Bottom–up characteristics such as stimulus saliency and the heterogeneity of search context also contribute to the computation in the priority map. The priority value influences the signal-to-noise ratio for selecting a target (see Zelinsky & Bisley, 2015, for detailed discussion of priority maps). On the low similarity trials, the priority assigned to the target was high because nontargets were dissimilar to the templates and were homogeneous. By contrast, the priority of the target was low on the high-similarity trials because nontargets were highly similar to the templates and were heterogeneous. The low signal-to-noise ratio on the high-similarity trials led to reduced N2pc while suppressing similar nontargets with high priority via lateralized alpha PLV approximately the time of selection. With stronger suppression, the N2pc became larger on the high-similarity trials.
Although we have been highlighting the distinct functions of alpha oscillation and N2pc during visual search as T-N similarity was varied, the distractor positive (Pd) that presents a positive going deflection contralateral to the distracting items is frequently reported in the literature of visual search (Hickey, Di Lollo, & McDonald, 2009). Several studies suggest that the Pd reflects a process to active suppression of a salient distractor (Feldmann-Wüstefeld & Vogel, 2019; Gaspelin & Luck, 2018a, 2018b; Sawaki, Geng, & Luck, 2012). We did not expect the occurrence of the Pd because no salient distractor was presented in search array. It is worth noting that the temporal relationship between the N2pc and Pd varied across different experiments. For example, ERP studies have revealed that the Pd may occur before (Feldmann-Wüstefeld & Vogel, 2019; Gaspelin & Luck, 2018a), after (Sawaki et al., 2012), or about the same time (Feldmann-Wüstefeld, Uengoer, & Schubö, 2015; Munneke, Fait, & Mazza, 2013) as the presence of the N2pc. Together with current results, these findings support the dynamic nature of top–down control mechanisms by which active suppression may interlink to the processes of target selection timely depending upon the task contexts or demands.
Taken together, our findings suggest the presence of dynamic and multiple competitive mechanisms during visual search. One mechanism of the search process is to allocate attention toward all possible target candidates and gate information by inhibition (Payne et al., 2013; Jensen & Mazaheri, 2010). When a search array appears, the neural populations in the visual cortex are tuned to the location of potentially relevant items, according to the current task goals (in our case, X and N). This process is influenced by the perceptual similarity of surrounding items within the search array. For example, when the T-N similarity is high, the target suffers strong competition from highly similar nontargets. This process thus allows for active inhibition or disengagement from irrelevant nontargets or distractors—the stronger the competition, the more strongly this information will need to be gated. We propose that this process corresponds to the spatially specific modulation of the alpha rhythm (Kuo et al., 2017; Payne et al., 2013; Klimesch, 2012; Händel et al., 2011; Rihs et al., 2009; Kelly et al., 2006; Thut et al., 2006; Worden et al., 2000). When targets and distractors compete more directly with one another, this spatiotopic modulation of sensory representations will need to be stronger. Another mechanism of search is to access the identity of target representations (Drisdelle & Jolicoeur, 2018; Aubin & Jolicoeur, 2016; Mazza, Turatto, & Caramazza, 2009). This mechanism allows for the reactivation of the neural ensembles coding for the target templates in working memory and the selection of the item that is best matched to the target template. The identification of target items and related attributes among sensory inputs can be accessed, and the perceptual decision can be made. A recent finding using MEG supports the account of dynamic and multiple competitive mechanisms (Kuo, Nobre, Scerif, & Astle, 2016). In their study, when participants searched for a target within an array, they first observed phase–phase coupling within the alpha band, between frontoparietal cortices and the visual areas. Importantly, this coupling corresponded to the spatial location of the target items, consistent with some early gating process. This alpha-band effect preceded the N2pc, as in the current study. Moreover, we showed that the amount of alpha phase adjustment was positively correlated with N2pc amplitude—further suggesting that these two mechanisms are functionally intertwined with each other. When T-N similarity is high, the target can be accessed and selected effectively if the similar nontargets can be suppressed beforehand or in parallel. Alongside these findings, the current results provide temporal markers for gating and spatiotopic access to the target representations during visual search.
In conclusion, we have isolated two electrophysiological effects that are sensitive to competition during visual search. We provided evidence showing that efficient search requires effective gating, which is reflected by alpha lateralization in phase adjustment. Along with that, target representations can be accessed and selected for making search decisions. Both processes are influenced by the competitive interaction between targets and nontargets.
This work was supported by a grant from the Ministry of Science and Technology, Taiwan (MOST 104-2628-H-002-002-MY3). The authors thank Duncan Astle for his comments and suggestions on an early version of the manuscript.
Reprint requests should be sent to Bo-Cheng Kuo, Department of Psychology, National Taiwan University, No. 1, Sec. 4, Roosevelt Road, 10617, Taipei, Taiwan, or via e-mail: email@example.com.