Abstract
The brain can organize elements into perceptually meaningful gestalts. Visual symmetry is a useful tool to study gestalt formation, and we know that there are symmetry-sensitive regions in the extrastriate cortex. However, it is unclear whether symmetrical gestalt formation happens automatically, whatever the participant's current task is. Does the visual brain always organize and interpret the retinal image when possible, or only when necessary? To test this, we recorded an ERP called the sustained posterior negativity (SPN). SPN amplitude increases with the proportion of symmetry in symmetry + noise displays. We compared the SPN across five tasks with different cognitive and perceptual demands. Contrary to our predictions, the SPN was the same across four of the five tasks but selectively enhanced during active regularity discrimination. Furthermore, during regularity discrimination, the SPN was present on hit trials and false alarm trials but absent on miss and correct rejection trials. We conclude that gestalt formation is automatic and task-independent, although it occasionally fails on miss trials. However, it can be enhanced by attention to visual regularity.
INTRODUCTION
The visual brain must organize and interpret the retinal image. We know many rules that govern perceptual organization (Wagemans et al., 2012; Hoffman, 1998), but the circumstances under which perceptual organization happens are unclear. Does the visual brain always organize and interpret the image to the fullest extent possible, or does it often leave aspects of the image uninterpreted (Lamme & Roelfsema, 2000; Mack & Rock, 1998; Treisman & Gelade, 1980)?
This research program requires an objective way to measure whether perceptual organization has happened or not. An ERP component called the sustained posterior negativity (SPN) provides this. Amplitude is more negative at posterior electrodes when participants view regular or symmetrical patterns where corresponding elements can be grouped into an organized whole (Makin, Wilton, Pecchinenda, & Bertamini, 2012; Jacobsen & Höfel, 2003). fMRI has identified symmetry-related BOLD responses in the extrastriate cortex (Van Meel, Baeck, Gillebert, Wagemans, & Op de Beeck, 2019; Keefe et al., 2018; Kohler, Clarke, Yakovleva, Liu, & Norcia, 2016; Sasaki, Vanduffel, Knutsen, Tyler, & Tootell, 2005; Tyler et al., 2005), and this extrastriate activity probably generates the SPN (Makin et al., 2016). We can therefore posit that if an SPN is recorded at the scalp, then perceptual organization has happened in brain (Bertamini, Silvanto, Norcia, Makin, & Wagemans, 2018).
The SPN is often similar when participants attend to stimulus regularity or to a secondary stimulus dimension such as color (Makin, Rampone, & Bertamini, 2015; Höfel & Jacobsen, 2007). This suggests symmetrical gestalts are formed automatically, whatever the current task is. However, Rampone, Makin, and Bertamini (2014) found that the SPN was abolished when participants read superimposed negative words, and the BOLD response to symmetry is sometimes reduced during color discrimination (Keefe et al., 2018; Sasaki et al., 2005). Furthermore, task manipulations could be more consequential when symmetrically arranged elements are embedded among randomly arranged noise elements (Gheorghiu, Kingdom, Remkes, Li, & Rainville, 2016; Palumbo, Bertamini, & Makin, 2015; Barlow & Reeves, 1979). Unlike perfect symmetry, such intermediate symmetries may not be extracted automatically in all tasks.
We measured the SPN at 20%, 40%, 60%, 80%, and 100% symmetry (as difference from the waveform elicited by the 0% symmetry, random pattern). This variable is termed PSYMM (Palumbo et al., 2015). There were five tasks, all of which involved a binary judgment (Figure 1). In the regularity task, participants judged whether patterns had some regularity or no regularity. In the color task, participants judged whether elements were light green or dark green. In the sound/color task they judged the congruence of the relationship between element color and the pitch of a simultaneous sound (congruent = high pitch and light green or low pitch and dark green, whereas incongruent = high pitch and dark green or low pitch and light green). Likewise, in the direction/color task, they judged the relationship between element color and direction of a central triangle. Finally, in the distribution task, participants judged whether elements were spread uniformly or concentrated around the periphery. The tasks had different cognitive and perceptual requirements, which could vary in the extent to which they take computational resources away from symmetrical gestalt formation. However, all secondary tasks were relatively easy and involved judgments about the pattern itself (rather than another object elsewhere on the screen, which is beyond the scope of this study).
Three predictions were preregistered (https://aspredicted.org/yy5gs.pdf). First, we predicted the SPN to scale with PSYMM in all tasks. This scaling was already found by Palumbo et al. (2015), and it is strongly predicted by psychophysics (Barlow & Reeves, 1979) and theoretical work (van der Helm, 2010). Second, we predicted the SPN to scale with how much the task resembled regularity discrimination: regularity = distribution > color = direction/color > sound/color. After all, in the distribution task, participants attended to the arrangement of elements, whereas in the sound/color task they were not even attending exclusively to the visual modality. Third, we predicted that the effect of task would be most pronounced at intermediate levels of PSYMM, such as 40% or 60%.
METHODS
Participants
There were 26 participants in each task and 130 participants in total (regularity: age 18–28 years, eight men, seven left-handed; color: age 18–51 years, four men, two left-handed; sound/color: age 18–24 years, three men, six left-handed; direction/color: age 18–28 years, five men, five left-handed; distribution: age 18–37 years, four men, four left-handed). All had normal or corrected-to-normal vision. The experiment had local ethics committee approval and was conducted in accordance with the Declaration of Helsinki (2008).
Apparatus
EEG was recorded using a BioSemi Active-2 system, with 64 scalp electrodes arranged according to the international 10–20 system. Band-pass filters were set at 0.16 and 100 Hz. Horizontal EOG and vertical EOG were monitored online for unwanted blinks and eye movements. In the regularity and color tasks, stimuli were presented on a 40 × 30 cm CRT monitor with 60-Hz refresh rate. Participants were seated 100 cm away from the monitor. For the other three tasks, our apparatus was upgraded, and stimuli were presented on a 51 × 29 cm LCD monitor and participants were seated 57 cm away. Stimuli sizes were consistent in terms of degrees of visual angle. A chin rest was used for head stabilization in all tasks. The experiment was programmed in Python using open source PsychoPy libraries (Peirce, 2007).
Stimuli
Example patterns are shown in Figure 1. There were 160 dots in each pattern. Patterns had four axes of symmetry, but the number of dots arranged around the axes varied from 0% (random) to 100% (perfect symmetry). This PSYMM variable increased in 20% increments. Further constraints prevented the elements from overlapping or falling at the center of the pattern. Elements were light green (PsychoPy RGB −1, 0.5, −1; 50 Cd/m2) or dark green (RGB −1, 0.2, −1; 25 Cd/m2). The background disk was midgray (RGB 0, 0, 0; 40 Cd/m2) and the rest of the screen was black (RGB −1 −1 −1; 0.15 Cd/m2). Note that PsychoPy RGB coordinates range from −1 to 1.
The gray background disk was 7.7° diameter. The central circular area where the elements could land was 7.16° diameter. Individual dot element diameter was 0.43°, but luminance was modulated with a Gaussian mask with a standard deviation of 1/6, so the visible dot-like element was approximately 0.21° diameter.
For the distribution task (Figure 1B), the 7.16° diameter zone where the elements fell was implicitly divided into a central disk and peripheral ring zones of equal area. In the uniform condition, there were 80 elements (50%) in each zone. In the peripheral condition, there were 32 (20%) elements in the central disk zone and 128 (80%) in the outer ring zone.
For all tasks, we generated 600 different stimuli in advance and saved them as .PNG files. The same 600 images were used for all participants within a task, but in a different randomized order. This approach allowed for rapid stimulus presentation without having to engage the computationally heavy stimulus generation algorithm on every trial. The same 600 images were used in regularity and color tasks, and another 600 images were used in sound/color and direction/color tasks. A third set of images was required for the distribution task.
Procedure
All trials began with a fixation baseline (1.5 sec), followed by pattern presentation (1.5 sec). After these, participants entered their binary judgment. In the regularity task, they classified each image as having “some regularity” or “no regularity” (see example in Figure 1C). In the color task, they classified images as “light” or “dark” (referring to the shade of green). In the sound/color task, participants saw patterns and simultaneously heard a low (200 Hz) or high (800 Hz) pitched beep. The task was to classify the relationship between element color and auditory pitch as either congruent or incongruent (congruent referred to either low pitched and dark green, or high pitched and light green, whereas incongruent referred to either low pitched and light green, or high pitched and dark green). The direction/color task was cognitively similar to the sound/color task. Participants classified the relationship between element color and direction of an up- or downward pointing small central triangle (Figure 5D) as congruent or incongruent (congruent referred to either a downward triangle and dark green, or upward triangle and light green, whereas incongruent referred to either downward triangle and light green, or upward triangle and dark green). Participants found the distinction between congruent and incongruent trials intuitive after some practice (see Walker, 2012, for discussion of such sensory correspondences). Finally, the distribution task required participants to judge whether all dots were spaced uniformly around the gray disk (uniform) or disproportionately around the periphery (outside).
The choice of tasks was partly motivated by previous research and partly by the need for internal coherence across the study. The color task had been used in previous work on automatic SPNs with 100% symmetry, and it allowed us to use same stimuli as the regularity task. The congruence tasks were designed to be like the color task, but further require another judgment about one additional perceptual dimension, either in the same or different sensory modality (e.g., sound or triangle direction). The distribution task was chosen because it focused attention on an aspect of spatial arrangement, but not on the symmetry itself. We wanted to test whether this would be sufficient to elicit SPNs with maximum amplitude (following similar experiments by Keefe et al., 2018).
In all five tasks, the judgments were entered after the stimulus had disappeared. Blue words reminded participants of the correct response mapping (Figure 1C). Participants entered their judgments with the A or L keys on a computer keyboard. There was no requirement to respond quickly. Participants attempted to maintain central fixation during baseline and presentation intervals and blink during the unspeeded response interval between trials.
In all tasks, there were 600 trials in total. The configuration of the elements in 300 trials was random. There were 60 trials at each of the five levels of PSYMM. PSYMM was fully crossed with other factors in the design. All tasks were thus designed so an ideal observer would enter each binary judgment an equal number of times at each level of PSYMM.
ERP Analysis
Continuous EEG data were processed offline using EEGLAB 13.4.4b (Delorme & Makeig, 2004) in MATLAB 2014b. Preprocessing conventions were chosen for consistency with previous work. Data were first referenced to the scalp average, low-pass filtered at 25 Hz, downsampled to 128 Hz to reduce file size, and then broken into epochs from −0.5 to +1.5 sec around stimulus onset, with a −200 msec prestimulus baseline. Blink and other large artifacts were removed from the epoched data with independent components analysis (ICA; Jung et al., 2000). In the regularity task, an average of 9.77 components were removed per participant (min = 4, max = 17). In the color task, average component removal was 10 (min 3, max 18). In the sound/color task, it was 9.42 (min 3, max 17); in direction/color task, it was 10.19 (min 0, max 17); and in the distribution task, it was 10.27 (min 1, max 18). After ICA, trials were rejected if amplitude were more extreme than ±100 μV at any electrode. Trial exclusion rate was similar across PSYMM levels, although there was some variation between tasks (regularity task, 2–3%; color, 4–5%; sound/color, 4–5%; direction/color, 9–10%; distribution, 6–8%).
Supplementary Material 1 (doi:10.17605/OSF.IO/WV6XB) examines the consequences of different preprocessing conventions using the regularity task data as an example. Among other things, we found that the grand-average ERPs were very similar when different trial exclusion limits were used and when EEG data were not cleaned with ICA. We note that ICA cleaning reduced trial exclusion by approximately 13.4%, without much altering the shape of the grand-average ERPs.
The SPN at each level of PSYMM was defined as the difference from random waves at a bilateral posterior electrode cluster [PO7 O1 O2 PO8], from 300 to 1000 msec poststimulus onset. The left [PO7 O1] and right [O2 PO8] pairs from this cluster were used when comparing SPN across hemispheres. These electrodes were chosen a priori.
We then analyzed global field power (GFP) as a function of PSYMM in each task. GFP was defined as the standard deviation of amplitudes across all 64 electrodes of the topographic difference map. If GFP effects parallel SPN effects, we can be more confident that the latter were not problematically dependent on our a priori electrode choice.
We conducted two further analyses of the same ERP data to visualize and confirm the observed effects of PSYMM and task. First, we used mass univariate analysis with the hierarchical linear modeling for EEG toolbox in MATLAB (LIMO; Pernet, Latinus, Nichols, & Rousselet, 2015; Pernet, Chauveau, Gaspar, & Rousselet, 2011). This applied a pairwise multilevel comparison to all electrodes and time points from −200 to +1000 msec. Each analysis compared the given PSYMM condition to the random condition. We used an alpha level of p < .01, but no spatiotemporal corrections for family-wise error rate. Second, we used the Randomization Graphical User Interface toolbox (RAGU; Koenig, Kottlow, Stein, & Melie-García, 2011) for two purposes. First, we used RAGU to assess whether topographies differed with PSYMM. The topographic ANOVA (TANOVA) procedure in RAGU uses randomization statistics to ascertain whether observed topographic differences are likely to occur due to chance. Second, we used RAGU to visualize the magnitude of differences between topographic maps on the first two dimensions from PCA.
Null Hypothesis Confirmation
p Values from ANOVA give the probability of obtaining the observed data given the null hypothesis (pD|H0) and not the probability of the null hypothesis being true given the observed data (pH0|D). This becomes a problem when theoretically interpreting a null result. We therefore used Bayesian alternatives to null hypothesis significance testing to supplement theoretically interesting nonsignificant effects (Masson, 2011). This provides estimates of pH0|D and allowed us to statistically confirm null results.
Open Science Policy
All codes for experimental presentation, stimulus, and EEG and behavioral analysis are freely available on Open Science Framework (doi:10.17605/OSF.IO/WV6XB), along with preprocessed EEG data and Supplementary Analyses 1 and 2. We are happy for other researchers to reanalyze our data or reuse our experiments or stimuli for any purpose.
RESULTS
Behavioral Results
Behavioral results from the five tasks are shown in Figure 2. Participants gave the correct response on nearly every trial in the color task and in the two congruence tasks (P correct near 100%). In the distribution task, P correct was not at ceiling, and there was a tendency to give the wrong answer more often at higher levels of PSYMM, F(1.823, 45.573) = 14.847, p < .001, partial η2 = .373.
In the regularity task, participants judged most random trials correctly (by reporting “no regularity”). When there was some regularity, P correct increased with PSYMM, F(1.676, 41.888) = 436.223, p < .001, partial η2 = .946. There was a significant increase in P correct with every increment of PSYMM (although performance was already near ceiling at 80%). At 40% PSYMM, performance was around chance level (P correct = 55.8%; not significantly greater than 50%, t(25) = 1.825, p = .08).
Sustained Posterior Negativity
The SPN is best represented as a difference from 0% symmetry (random) wave, as in the second row of Figure 3. There was a strong effect of PSYMM on SPN amplitude in all five tasks. However, the SPN was selectively enhanced in the regularity task (Figure 3, first column). These effects can also be seen in the topographic difference maps in Figure 4, where the SPN appears as blue at posterior electrodes.
For statistical analysis, SPN amplitude in the 300–1000 msec interval was examined with mixed ANOVA (5 PSYMM × 5 Task). Mean SPN amplitudes are shown in Figure 5C, with 95% confidence intervals. There were main effects of PSYMM, F(3.174, 396.812) = 123.961, p < .001, partial η2 = .498, and Task, F(4,125) = 8.851, p < .001, partial η2 = .221, and a PSYMM × Task interaction, F(12.698, 396.812) = 3.219, p < .001, partial η2 = .093.
The PSYMM × Task interaction was followed up with sub-ANOVAs. When regularity task was removed from the analysis, the SPN results were characterized by a significant main effect of PSYMM, F(3.098, 309.803) = 78.998, p < .001, partial η2 = .441, but no main effect of Task, F < 1, ns, pH0|D = 0.999, and no PSYMM × Task interaction, F(9.294, 309.803) = 1.316, p = .226, pH0|D > 0.999. Meanwhile, the regularity task differed from each of the others when pairs of tasks were analyzed, both in terms of main effects and interactions: smallest effect, F(3.037, 151.856) = 4.461, p = .005, partial η2 = .082. The regularity task thus statistically stands out from the other four.
In all our tasks, PSYMM was crossed with another factor, such as element color, congruence, or distribution. The SPN never interacted with these factors (see Supplementary Material 2, doi:10.17605/OSF.IO/WV6XB). For instance, PSYMM did not interact with Greenshade, although this generated an independent ERP at posterior central electrodes. This is consistent with previous work, where the extrastriate symmetry response was independent of low-level luminance and color properties, as long as contrast was far above threshold (Martinovic, Jennings, Makin, Bertamini, & Angelescu, 2018).
We predicted that the effect of Task would be larger at intermediate levels of PSYMM, such as 40% or 60%. However, our results were not consistent with this hypothesis (Figure 5C). Instead, the SPN enhancement in the regularity task was equivalent at all levels of PSYMM above 20%. Indeed, when the 20% PSYMM condition was excluded from the analysis, there was a main effect of Task, F(4, 125) = 11.368, partial η2 = .267, but no PSYMM × Task interaction, F(9.907, 309.607) = 1.743, p = .071, pH0|D > 0.999.
There are several ways to describe the magnitude of the task effect. Amplitude was around 0.75–1.61 μV larger in the regularity task than in the average of the other four (depending on PSYMM; Figure 5C). In the 100% PSYMM condition, this constituted an average of 60% SPN increase when participants attended to regularity. This could equally be described as an average of 36% SPN decrease when participants attended to something other than regularity.
Finally, we note that, in the regularity task, there was a significant SPN at 40% PSYMM, t(25) = −4.675, p < .001. In the color task and distribution task, the first significant SPNs were at 60% (t(25) = −2.952, p = .007, t(25) = −3.350, p = .003), whereas in the sound/color task and direction/color task, the first significant SPNs were at 80% (t(25) = −4.603, p < .001; t(25) = −4.298, p < .001). Therefore, attention to regularity made the critical difference between the presence and absence of an SPN at 40% PSYMM.
P1 and N1 Peaks
The top row in Figure 3 shows grand-averaged ERP waves in each task. Among other things, it can be seen that P1 was reduced in the distribution task, whereas the N1 was enhanced in the regularity task. To explore these effects, we obtained P1 and N1 peak amplitude during the 130–180 msec and 180–250 msec windows (Figure 5A and B).1 There was only a borderline effect of PSYMM on P1 peak amplitude, F(4.339, 542.377) = 2.291, p = .053, partial η2 = .018, pH0|D > 0.999. However, there was a main effect of Task, F(4, 125) = 3.764, p = .006, partial η2 = .107 (Figure 5A). Conversely, N1 peak amplitude increased with PSYMM, F(4.194, 524.249) = 17.878, p < .001, partial η2 = .125, and differed between tasks, F(4, 125) = 5.796, p < .001, partial η2 = .156 (Figure 5B).
Source Localization
We used low-resolution electromagnetic tomography (LORETA) to estimate the anatomical location of SPN generators (Pascual-Marqui, Michel, & Lehmann, 1994). To obtain the strongest SPN signal, we used the grand-averaged 100% PSYMM SPN, collapsed across all tasks (n = 130). As expected, LORETA found sources in the bilateral extrastriate cortex, with some right hemisphere bias (Figure 4D). This is consistent with fMRI investigations, which have consistently found symmetry activations in these brain areas (Van Meel et al., 2019; Sasaki et al., 2005). It is also consistent with more sophisticated EEG source analysis reported in an earlier work (Kohler et al., 2016). The right lateralization of the SPN was also confirmed statistically (Supplementary Material 2, doi:10.17605/OSF.IO/WV6XB).
Global Field Power
GFP was defined as standard deviation of 64 amplitudes across a topographic difference map. We obtained GFP at each time point. The third row of Figure 3 shows that GFP effects paralleled the SPN effects. GFP increased dramatically at around 200 msec and scaled with PSYMM thereafter. Most importantly, GFP was stronger in the regularity task than the other four tasks (see Supplementary Material 2, doi:10.17605/OSF.IO/WV6XB). The similarity between GFP and SPN results demonstrates that the SPN results were not problematically dependent on a priori electrode choice.
Topographic Analysis
Next, we examined SPN topography using TANOVA (Koenig et al., 2011) and found a strong effect of PSYMM (p < .001). This reconfirms that the topographies changed with PSYMM. More interestingly, Figure 4B shows multidimensional scaling (MDS) plots from each task. Here the grand-averaged topographies are represented as points, with x- and y-axes representing the first two dimensions from PCA. If data points are close together in this 2-D space, then the corresponding topographies are similar. If data points are far apart, the corresponding topographies are disparate (see Koenig et al., 2011, for details). The schematic topographies in Figure 4C are not data but represent the distribution of the first principal component. Data points near the left are more like the left schematic topography (i.e., more positive at posterior electrodes), and data points near the right are more like the right schematic topography (i.e., more negative at posterior electrodes). As expected, topographies from each level of PSYMM are spread along the first dimension (x-axis), with far less variability along the second dimension (y-axis). Furthermore, it can be seen that the topographies are uniquely spread out across the x-axis in the regularity task.
Mass Univariate Analysis
The SPN results were also examined and visualized with mass univariate analysis (Pernet et al., 2011, 2015). Multilevel pairwise comparisons contrasted each PSYMM level to random at each electrode and time point (Figure 6). For example, in the lower row, each point represents a t score from a pairwise comparison between 100% symmetry and random at a particular time point and from a particular electrode. The SPN appears as green/blue at posterior electrodes (see arrows at the bottom right). Again, we see the SPN increased with PSYMM (rows) and was selectively enhanced in the regularity task (left column).
SPN from Correct and Incorrect Trials of the Regularity Task
In the regularity task, participants did not always give the correct answer at intermediate levels of PSYMM (Figure 2). We assume participants applied an internal decision threshold to the analogue symmetry signal to make a binary judgment (no regularity or some regularity). If the analogue signal exceeded the internal threshold, they reported some regularity (Kohler, Cottereau, & Norcia, 2018; Palumbo et al., 2015).
In the 20% trials, the symmetry signal rarely exceeded threshold, so these patterns were nearly always judged incorrectly as having “no regularity” (miss). Conversely, In the 80% and 100% trials, the symmetry signal nearly always exceeded threshold, so these patterns were nearly always judged correctly as having “some regularity” (hit). Likewise, we can distinguish between random trials where participants correctly reported “no regularity” (correct rejection) and random trials where they incorrectly reported “some regularity” (false alarm).
First, we consider the distinction between correction rejection and false alarm trials, where the patterns were random. In the false alarm trials, there was a low-amplitude SPN wave (Figure 7 top row). It is possible that images that generated false alarm SPN had some undesigned symmetrical-looking features, which might be discernable through trial-by-trial image analysis. Furthermore, features of the trial sequence, such as many random patterns in a row, could have led to occasional false alarm SPNs.2
Next, we consider the distinction between hit and miss trials, where PSYMM was > 0 (Figure 7, rows 2–4). There was a clear SPN on hit trials but not the miss trials. Interestingly, the P1 and N1 peaks were very similar on hit and miss trials. This suggests that our participants had not merely closed their eyes and thus guessed incorrectly or fixated elsewhere on the screen throughout the trial on miss trials.
Separate repeated-measures ANOVAs confirmed these impressions (2 judgment [correct, incorrect] × 4 PSYMM [random, 20%, 40%, 60%]). For P1 and N1 peaks (Figure 8A and B) there were no main effects or interactions: largest effect, F(1.966, 49.158) = 1.086, p = .345, pH0|D = 0.934. In contrast, for the SPN, there was a Judgment × PSYMM interaction, F(2.208, 55.200) = 7.878, p = .001, partial η2 = .240, because there was a main effect of PSYMM on the correct trials, F(3, 75) = 16.981, p < .001, partial η2 = .404, but not on the incorrect trials, F(1.965, 49.127) = 1.024, p = .366, pH0|D = 0.936 (Figure 8C). Three conditions produced a significant SPN (Figure 8C). These were the incorrect (false alarm) random trials, t(25) = −2.453, p = .021; correct (hit) at 40% trials, t(25) = −4.441, p < .001; and correct (hit) 60% trials, t(25) = −8.463, p < .001.3 These conditions also generated significant differences between correct and incorrect trials (see stars on Figure 8C).
DISCUSSION
As expected, we found an SPN response to symmetry, which increased with PSYMM (Palumbo et al., 2015). Surprisingly, the SPN was similar in all tasks except the regularity task, where it was selectively enhanced. Indeed, at 40% PSYMM, attention to regularity made the difference between the presence and absence of an SPN.
It is remarkable that the four non-regularity-related tasks produced such similar SPN waves. By design, the tasks were different in several respects. In the distribution task, participants attended to arrangement of the elements, so we predicted large SPNs here. At the other extreme, in the sound/color task, participants partly attended to the auditory modality, so we predicted small SPNs here. However, the expected task differences were not found.
The same basic pattern of results was confirmed with GFP, TANOVA, and mass univariate analysis. We thus conclude that there is a default, task-independent parametric response to symmetry in the extrastriate cortex. However, this default response can be upregulated when necessary, for example, when the task is to find symmetry in random noise.
The apparent automaticity of the symmetry detection, at least during nondemanding secondary tasks, is potentially adaptive. Symmetry is a visual cue used to find objects against a visually cluttered background (Machilsen, Pauwels, & Wagemans, 2009). Perhaps symmetry detection mechanisms are nearly always online because it is nearly always important to parse the visual scene?
Hemispatial neglect caused by right dorsal stream lesions can impair vertical symmetry discrimination because patients ignore the left side of the stimulus. However, symmetry still aids figure-ground segmentation in these patients (Driver, Baylis, & Rafal, 1992). This again suggests symmetry can be processed automatically (and unconsciously) in the ventral stream. This is consistent with our results, where symmetry was processed automatically in all tasks.
Generalizability of Conclusions
Of course, we only explored four nonregularity tasks here. We cannot assume the same results would be obtained across all conceivable secondary tasks. For instance, SPN might be greatly reduced if the secondary task were very demanding and used more attentional resources, as in the experiments by Mack and Rock (1998). However, we note that SPN was nearly identical in our easy color task, in the more difficult distribution task (where performance was not a ceiling), and in our congruence tasks where information had to be integrated across two perceptual dimensions. This suggests that the SPN is robust to a wide range of task manipulations, although the boundaries of task independence remain to be established.
Likewise, our experiments only tested automaticity of symmetry perception, not the automaticity of other kinds of perceptual organization. The SPN can also be generated by other gestalts such as glass patterns (Makin et al., 2016) and perhaps by recognizable real objects (Martinovic, Mordal, & Wuerger, 2011). This indirectly suggests that our results may generalize beyond symmetry. However, automaticity is unlikely to be a universal law of perceptual organization, which involves a heterogenous set of more or less automatic subprocesses (Barbot, Liu, Kimchi, & Carrasco, 2018).
There is probably an important distinction between the less and more computationally demanding types of perceptual organization. Perhaps only the less demanding types happen automatically across secondary tasks. Extraction of symmetry from the image is an example of this. In contrast, more demanding types of perceptual organization are probably not automatic and often fail during secondary tasks. Examples include the formation of allocentric, view invariant object representations (Makin et al., 2015), and formation of an object representations from temporally separated images (Rampone, Makin, Tatlidil, & Bertamini, 2019).
SPN and Binary Judgments
There was a systematic relationship between recorded brain responses and behavior in our regularity task. When the pattern was random, participants usually gave the correct answer and reported “no regularity” (a correct rejection). However, they occasionally gave an incorrect answer and reported “some regularity” (a false alarm). There was a weak SPN on these false alarm trials. When the pattern was symmetrical, participants sometimes correctly reported “some regularity” (a hit) or incorrectly reported “no regularity” (a miss). There was a parametric SPN on the hit trials, but no SPN on miss trials. On miss trials, it seems that perceptual organization failed, and no symmetry representation was generated in the extrastriate cortex, despite availability the symmetry in the distal stimulus. This result is interesting because P1 and N1 components of the visual evoked potential (VEP) were present on miss trials, so participants were likely processing the stimulus during the early part of the trial (and not, for example, closing their eyes or fixating elsewhere throughout).
Although the extrastriate symmetry response sometimes failed, we do not think this means it should be considered as nonautomatic. After all, even an automatic perceptual response will not be elicited by the stimulus with 100% reliability.
An ERP that is present on both hit and false alarm trials, but absent on both miss and correct rejection trials, would be typically associated with binary decision-making or behavioral responses rather than perception. However, we argue that the SPN is a measure of perceptual organization, not decision-making or behavior. First, SPN amplitude increased with PSYMM on hit trials, although the behavioral response was identical. Second, the false alarm SPN was much weaker than the 60% hit SPN, even though the response was again identical. Third, SPN amplitude was related to PSYMM in a linear fashion, even though the frequency of “some regularity” responses nearly plateaued at 80%. All these observations are explicable if we assume that participants applied an internal threshold to the analogue symmetry signal and reported “some regularity” if the symmetry signal exceeded threshold. The SPN is generated by the analogue symmetry signal, not the decision-making processes or behavior (see Kohler et al., 2018, for further analysis of decision-making in symmetry perception).
Why did we observe a false alarm SPN on some random trials in the regularity task? There are two obvious explanations. First, our stimulus construction algorithm may have occasionally introduced some regularity into the random patterns by chance. In this case, the term “false alarm” is a misnomer: The visual system detected slight regularity present in the stimulus. Second, internal neural fluctuations may have led to super-threshold symmetry activations on some occasions. These are genuine false alarms. The relative contribution of each effect to the observed “false alarm” SPN is uncertain. However, neither phenomena fundamentally challenges our claim that the SPN is a measure of automatic perceptual organization.
Finally, we note that the participants in our regularity task were searching for intermediate symmetry in noisy displays, and the weak symmetry signal was often near the threshold of discriminability. These task parameters are likely to maximize responsiveness to any residual regularity in randomly generated images. This could be tested in future work: The false alarm SPN on random trials might disappear if the task required mere discrimination of random from 100% symmetry, with no ambiguous intermediate levels of PSYMM.
VEP and Task
Analysis of P1 and N1 peaks was also interesting in other ways. N1 was much larger in the regularity task than the others. Previous work has found that visual processes at the N1 latency mediate spatial integration (Shpaner, Molholm, Forde, & Foxe, 2013). Maybe spatial integration was more pronounced in the regularity task, where it was required? Furthermore, N1 increased with PSYMM. This replicates previous research where later SPN effects are already weakly present at N1 (Makin, Rampone, Pecchinenda, & Bertamini, 2013). Further work is needed to clarify early visual processing stages in gestalt formation, which happen at the N1 latency. Conversely, P1 was not modulated by PSYMM, presumably because this peak indexes early visual onset detection, before the extrastriate symmetry response begins.
Right Lateralization of the SPN
The SPN was slightly stronger over the right hemisphere than the left, particularly in the regularity and color tasks, and source localization also found a predominantly right hemisphere symmetry activation. Although there is no clear explanation for why this right lateralization should be stronger in some tasks than others, it has been observed previously (Bertamini & Makin, 2014). Moreover, converging evidence comes from three other methodologies. First, TMS disruption of the right lateral occipital complex selectively impairs regularity discrimination (Bona, Herbert, Toneatto, Silvanto, & Cattaneo, 2014). Second, regularity presented to the right hemisphere is discriminated more quickly (Verma, Van der Haegen, & Brysbaert, 2013). Third, regularity can be more reliably decoded from right lateral occipital complex multivoxel patterns (Van Meel et al., 2019). We conclude that the extrastriate symmetry response is weakly right-lateralized, although there is certainly a symmetry response in the left hemisphere too (Wright, Makin, & Bertamini, 2017).
Conclusions
When does the visual brain expend computational resources to compare correspondence across locations and extract symmetry from the image? We found a default parametric response to symmetry in the extrastriate cortex, which was similar across five very different tasks. This suggests the brain always engages some perceptual organization to detect symmetry. However, this default symmetry response was selectively enhanced when regularity was task relevant. We conclude that symmetrical gestalt formation is (1) largely task independent, (2) can be enhanced by attention to symmetry, but (3) fails on a minority of trials, so there is no extrastriate activation and symmetry is missed.
Acknowledgments
This project was part funded by an Economic and Social Research Council grant award to Alexis D. J. Makin (ES/S014691/1). We would like to thank project students Zaynah Ahmed, Luke Bennett, Eleanor Dickens, Alice Newton-Fenner, and Kasia Gmaj for helping with data collection.
Reprint request should be sent to Alexis D. J. Makin, Department of Psychological Sciences, Eleanor Rathbone Building, University of Liverpool, Liverpool, L69 7ZA, United Kingdom, or via e-mail: [email protected].
Notes
It is unclear why the VEP was delayed by approximately 33 msec in these experiments. This was at least partly due to timing differences between older and newer apparatus. However, we are not theoretically interpreting P1 or N1 latency, just amplitude.
After the regularity task, our participants filled in the Oxford–Liverpool Inventory of Feelings and Experiences (O-LIFE) questionnaire, which measures psychotic symptoms in the normal population (Mason & Claridge, 2006). There was no hint of a correlation between the amplitude of false alarm SPN and scores of the unusual experiences subscale, which measures hallucination proneness (r = .04). Future researchers interested in clinically relevant individual differences in vision should be aware of this null result.
This analysis should be treated with some caution, because the number of trials at 20% correct and 60% incorrect conditions was low (average 12 and 14). This is not considered enough to generate reliable ERP waves (see noise in the prestimulus baseline). However, this did not obscure systematic effects, and the P1 and N1 were similar.