Abstract

Throughout the brain, information from individual sources converges onto higher order neurons. For example, information from the two eyes first converges in binocular neurons in area V1. Some neurons are tuned to similarities between sources of information, which makes intuitive sense in a system striving to match multiple sensory signals to a single external cause—that is, establish causal inference. However, there are also neurons that are tuned to dissimilar information. In particular, some binocular neurons respond maximally to a dark feature in one eye and a light feature in the other. Despite compelling neurophysiological and behavioral evidence supporting the existence of these neurons [Katyal, S., Vergeer, M., He, S., He, B., & Engel, S. A. Conflict-sensitive neurons gate interocular suppression in human visual cortex. Scientific Reports, 8, 1239, 2018; Kingdom, F. A. A., Jennings, B. J., & Georgeson, M. A. Adaptation to interocular difference. Journal of Vision, 18, 9, 2018; Janssen, P., Vogels, R., Liu, Y., & Orban, G. A. At least at the level of inferior temporal cortex, the stereo correspondence problem is solved. Neuron, 37, 693–701, 2003; Tsao, D. Y., Conway, B. R., & Livingstone, M. S. Receptive fields of disparity-tuned simple cells in macaque V1. Neuron, 38, 103–114, 2003; Cumming, B. G., & Parker, A. J. Responses of primary visual cortical neurons to binocular disparity without depth perception. Nature, 389, 280–283, 1997], their function has remained opaque. To determine how neural mechanisms tuned to dissimilarities support perception, here we use electroencephalography to measure human observers' steady-state visually evoked potentials in response to change in depth after prolonged viewing of anticorrelated and correlated random-dot stereograms (RDS). We find that adaptation to anticorrelated RDS results in larger steady-state visually evoked potentials, whereas adaptation to correlated RDS has no effect. These results are consistent with recent theoretical work suggesting “what not” neurons play a suppressive role in supporting stereopsis [Goncalves, N. R., & Welchman, A. E. “What not” detectors help the brain see in depth. Current Biology, 27, 1403–1412, 2017]; that is, selective adaptation of neurons tuned to binocular mismatches reduces suppression resulting in increased neural excitability.

INTRODUCTION

It remains an important challenge in neuroscience to understand how the brain combines a pair of 2-D retinal images to support 3-D perception. Classically, this problem has been framed as one of matching features between the two eyes, that is, solving the “stereoscopic correspondence problem,” so that the depth of objects can be triangulated (Julesz & Chang, 1976; Marr & Poggio, 1976). This problem is nontrivial, as the number of “false matches” (i.e., correspondences between features that do not originate from the same object) rapidly increases with the number of to-be-matched elements.

Random-dot stereograms (RDSs) are frequently used to investigate binocular vision because of their ability to divorce information about 2-D form from differences between the two eyes. These stimuli are composed of many self-similar features, potentially posing a severe challenge to establishing binocular correspondence. The classic framework for understanding stereopsis is to find correspondence by considering a range of potential disparities and selecting the offset that maximizes the image similarity between the two eyes (Fleet, Wagner, & Heeger, 1996; Ohzawa, DeAngelis, & Freeman, 1990). This makes intuitive sense; however, some disparity-selective neurons in V1 appear poorly optimized for such a computation in that they respond maximally to different images presented on the two retinae (Read & Cumming, 2007; Cumming & Parker, 1997). Moreover, binocular neurons can show tuning to images that are difficult to imagine being produced in the real world. A prime example of this is the test of neural function with anticorrelated RDSs (aRDSs) in which the polarity of image features is reversed between the two eyes. Unlike correlated RDSs (cRDSs), viewing aRDS does not support reliable depth perception; nevertheless, some disparity-selective neurons in V1 respond strongly to these stimuli. Despite empirical evidence supporting the existence of these neurons in macaques (Janssen, Vogels, Liu, & Orban, 2003; Tsao, Conway, & Livingstone, 2003; Cumming & Parker, 1997) and humans (Katyal, Vergeer, He, He, & Engel, 2018; Kingdom, Jennings, & Georgeson, 2018), their functional role remains opaque.

Recent theoretical work suggested a potential explanation for neurons tuned to mismatched binocular features. In their binocular likelihood model of stereopsis, Goncalves and Welchman (2017) suggested a simple decoding rule for binocular neurons: Information about depth can be read out from a population of binocular neurons where the decoding scheme is based on the cross-correlation between the encoding receptive fields. Under this scheme, the activity of a binocular neuron can lead to increased excitation for a particular depth interpretation or drive suppression of a specific depth estimate. By reading out a population of binocular neurons, it is possible to derive a likelihood estimate of the depth of the scene. This provides a plausible explanation for why neurons should respond to binocular correspondences that do not relate to a single physical object in the environment. In particular, the “what not” responses of binocular neurons can be used to drive suppression of unlikely interpretations of the scene. Despite this theoretical promise, there is little empirical evidence for the role of “what not” responses in the human visual system.

The idea that binocular mismatches are used to drive suppression in visual cortex yields a distinct prediction concerning the balance of excitation and inhibition following a period of adaption. In particular, adapting the responses of units that drive suppression should lead to less inhibition, thereby increasing the net excitation of the cortex. To investigate the role of “what not” responses within the visual cortex, here we use electroencephalography to measure human observers' brain activity during and after prolonged viewing of aRDS. Specifically, we measure steady-state visually evoked potentials (SSVEP) in response to cRDS and aRDS, following adaptation to either aRDS or cRDS. We find that, following adaptation to aRDS, SSVEP amplitude in response to cRDS increases relative to a preadaptation baseline. These results are consistent with the idea that “what not” responses play a suppressive role in supporting stereopsis; that is, selective adaptation of “what not” responses reduce suppression, resulting in increased neural excitability.

METHODS

Participants

Observers were recruited from the University of Cambridge, had normal or corrected-to-normal vision, and were screened for stereo deficits. Thirty-three right-handed human adults (eight men, age = 25.2 ± 4.8 years) participated in the main experiment; however, two were not included in the analysis: One was unable to see depth in the stimulus, and a hardware fault stopped acquisition midway through the experiment for the other. Of the 31 participants included in the analysis, 22 completed all experimental conditions; the remaining nine did not participate in the baseline condition. Twenty-two right-handed human adults (five men, age = 25.4 ± 4.6 years) participated in the control experiment. Experiments were approved by the University of Cambridge Ethics Committee; all observers provided written informed consent.

The sample sizes used in the experiments were selected based on previous studies using similar techniques to study stereopsis (Cottereau, McKee, Ales, & Norcia, 2012; Cottereau, McKee, & Norcia, 2012).

Apparatus and Stimuli

Stimuli were generated in MATLAB (The MathWorks, Inc.) using Psychophysics Toolbox and Eyelink Toolbox extensions (Cornelissen, Peters, & Palmer, 2002; Brainard, 1997; Pelli, 1997; see psychtoolbox.org/). Binocular presentation was achieved using a pair of Samsung 2233RZ LCD monitors (120 Hz, 1680 × 1050) viewed through mirrors in a Wheatstone stereoscope configuration. The viewing distance was 50 cm, and participant head position was stabilized using an eye mask, headrest, and chin rest. Eye movement was recorded binocularly at 1 kHz using an EyeLink 1000 (SR Research Ltd.).

Adaptation stimuli consisted of RDS (12° × 12°) on a mid-gray background surrounded by a static grid of black and white squares intended to facilitate stable vergence. Dots in the stereogram followed a black or white Gaussian luminance profile, subtending 0.07° at half maximum. There were 108 dots/deg2, resulting in ∼38% coverage of the background. In the center of the stereogram, four wedges were equally distributed around a circular aperture (1.2°), each subtending 10° in the radial direction and 70° in polar angle, with a 20° gap between wedges (Figure 1A). The wedge formation was used to perceptually accentuate the near/far regions from the surrounding zero disparity surface. Dots constituting the wedges were offset by 10 arcmin between the left and right eyes, and the remaining dots had zero offset. This disparity was large enough to clearly distinguish the near/far surface from the surrounding region while still being comfortable to stereoscopically fuse for a prolonged period. Stimuli were presented for 1 sec and separated by 1-sec interstimulus intervals consisting of only the background and fixation cross. On each presentation, we applied a random polar rotation to the set of wedges such that the disparity edges of the stimuli were in different locations for each stimulus presentation (i.e., a rigid body rotation of the four depth wedges together around the fixation point). In every eight presentations, we reversed the sign of the disparity of the wedges (crossed and uncrossed; Figure 1C). We balanced the presentation of near and far disparities to ensure refixation on the zero disparity reference plane, rather than promoting fixation away from the reference plane. At a given time point, all wedges were presented the same disparity. In the center of the wedge field, we presented a fixation square (side length = 1°) paired with horizontal and vertical nonius lines.

Figure 1. 

Adaptation and test stimuli used in the experiment. Diagram of the depth arrangement of the (A) adaptation and (B) test stimuli. (C) Example stimuli used in the experiment, designed for red-cyan anaglyph viewing.

Figure 1. 

Adaptation and test stimuli used in the experiment. Diagram of the depth arrangement of the (A) adaptation and (B) test stimuli. (C) Example stimuli used in the experiment, designed for red-cyan anaglyph viewing.

Test stimuli were similar to adaptation stimuli, except that, instead of rotating wedges, an annulus was used (Figure 1B, C). The depth sign of the annulus was reversed at 4-Hz frequency, whereas all dots were regenerated at 20-Hz frequency.

Procedure

Participants underwent three different runs: an initial baseline run, followed by correlated and anticorrelated runs (counterbalanced across participants). The baseline run consisted of five blocks of test stimulus presentations, each lasting 25 sec, separated by 20-sec blank interblock intervals. Correlated runs consisted of 12 adaptation blocks, each followed by a 2-sec blank interblock interval before a test block. Adaptation blocks consisted of 64 sec of adaptation stimuli presentations (32 presentations total), and test blocks were identical to those in the baseline run. Anticorrelated runs were identical to correlated runs, except that the polarity of all dots in the left eye was reversed during the adaptation blocks. During adaptation blocks, we instructed participants to fixate on the central fixation square while performing a Vernier discrimination task (Preston, Li, Kourtzi, & Welchman, 2008). During test blocks, we instructed observers to maintain fixation and limit blinks. cRDSs were used as the test stimulus for all conditions, providing an equal test of the effects of adaptation. Although viewing either aRDS or cRDS can produce an electrophysiological response, cRDSs were used as the test stimulus as they evoke a larger response than aRDSs (Petrig, Julesz, Kropfl, Baumgartner, & Anliker, 1981; Braddick et al., 1980), providing a better signal-to-noise ratio (SNR) and thus a more sensitive measure of neural change resulting from adaptation.

EEG

Electroencephalography data were acquired from all 33 participants with a 64-channel cap (BrainCap, Brain Products GmbH). Data were recorded using BrainVision Recorder software. Caps were fitted with 61 Ag/AgCl electrodes positioned according to the standard 10–20 system (Fp1 Fp2 F3 F4 C3 C4 P3 P4 O1 O2 F7 F8 T7 T8 P7 P8 Fz Cz Pz IO FC1 FC2 CP1 CP2 FC5 FC6 CP5 CP6 FT9 FT10 F1 F2 C1 C2 P1 P2 AF3 AF4 FC3 FC4 CP3 CP4 PO3 PO4 F5 F6 C5 C6 P5 P6 AF7 AF8 FT7 FT8 TP7 TP8 PO7 PO8 Fpz CPz POz Oz). Data were acquired with a reference electrode at FCz. Electrooculograms were also acquired, using two pairs of bipolar electrodes placed horizontally and vertically around the left eye. Data were high-pass filtered online at 0.1 Hz and acquired with a 1-Hkz sampling rate. Temporal markers were sent from the stimulus presentation computer to mark the onset of the stimulus. These timings were validated using a pair of photodiodes attached to the two stimulus presentation screens.

Preprocessing and analyses were performed in MATLAB using the EEGLAB toolbox (Delorme & Makeig, 2004) and custom in-house scripts. Data were first filtered offline with a 1-Hz high-pass and a 40 Hz low-pass filter. For the SSVEP analysis, each epoch was extracted around the test duration to include a period of 29 sec (2 sec before the first test stimulus onset to 2 sec after the offset of the final test stimulus); thus, there were 12 epochs per adaptation condition and five in the baseline, all of which were included in the analysis. For the ERP analysis, epochs were extracted around the stimulus onset to include a 1-sec prestimulus and 1-sec poststimulus period (384 epochs per adaptation condition). The poststimulus period, therefore, did not include any data from the next stimulus presentation. Epochs used in the ERP analysis were visually inspected, and artifactual epochs were rejected (excluding eye movements). For one participant, two channels (TP9 and TP10) were interpolated. All data were re-referenced to an average reference across all channels, and then independent component analysis (ICA) decomposition was applied for the purpose of artifact identification. Resultant ICA components were visually inspected before rejection. Only components reflecting eye movements and other likely muscle artifacts were removed. These components were identified by characteristic features in the component time course and power spectrum, in addition to their frontal topography.

There were 12 blocks in each adaptation run and five blocks in the baseline run; thus, we matched the length of data in each calculation by including data from the first half of the adaptation epochs (4–13.5 sec) and 22.8 sec of data from the baseline epochs (2–24.8 sec). That is, we included 12 blocks × 9.5 sec of data from the adaptation runs (144 sec total) and five blocks × 22.8 sec of data from the baseline run (144 sec total). We avoided including data immediately from stimulus onset to avoid contamination by any onset evoked potential. Fourier transformation of the data was performed on the full concatenated data recording; no window was applied to the data before transformation.

Eye Tracking

Owing to the bespoke experimental setup (i.e., recording eye position from behind one-way mirrors in a haploscope), the eye tracker would occasionally fail to track participants' eyes for an entire block. To draw within-participant comparisons, we only included the participants for whom data were available for both experimental conditions in the eye tracking analysis (n = 21). Before analysis, eye movement data were screened to remove noisy and/or spurious recordings. To remove spurious significant differences in the time course between conditions, a cluster correction was applied. Clusters were defined by the sum of their constituent (absolute) t values and compared with a null hypothesis distribution of clusters produced by shuffling the condition labels (1000 permutations). Clusters below the 95th percentile of the null hypothesis distribution were disregarded.

Significance Testing

The significance of differences between data from different conditions was assessed using the repeated-measures ANOVA and the paired t test, and the significance of differences between data from different experiments was assessed using the independent t test. The normality assumption was tested with the Shapiro–Wilk test of normality. Excluding the time-series analyses, there were 16 t tests performed. For six of these, the raw data violated the assumption of normality. Similarly, the raw data used in the repeated-measures ANOVA violated the assumption of normality. Thus, before assessing the differences between groups for these tests, we normalized the data using a log transform. We report the results of the normalized data, although for all cases the pattern of results was the same for the raw data.

RESULTS

SSVEP Analysis of Test Stimuli

If mismatches between the two eyes evoke inhibitory activity, we would expect that selectively reducing the responsiveness of the neural mechanisms that respond to mismatches (through adaptation) would lead to less inhibitory activity and thus relatively more excitability. To test this idea, observers were initially adapted to binocular mismatches by viewing aRDS for a prolonged period (64 sec). Then, following adaptation, participants viewed a cRDS composed of black and white blobs depicting an annulus that was either near or far relative to the background. We used the ERP evoked by changing the depth sign of the annulus (from near to far or far to near) as an index of stereoscopic related activity and the ERP evoked by refreshing the positions of the dots as an index of non-stereoscopic-related activity. Similar to previous work (Cottereau, McKee, Ales, et al., 2012; Cottereau, McKee, & Norcia, 2012; Cottereau, McKee, Ales, & Norcia, 2011), we rapidly changed the stimuli at two frequencies, that is, 4 Hz for depth sign and 20 Hz for dot refresh, producing two distinct SSVEPs. We measured the activity evoked by the stimulus changes by performing a Fourier transform on the data, converting it from the time domain to the frequency domain and taking the SNR between the peak at the target frequencies (4 and 20 Hz) and the baseline noise in the spectrum (from bins either side of the target frequency). For comparison, we also measured observers' SSVEP in response to the test stimulus following adaptation to cRDS and without prior adaptation (baseline).

To guide electrode selection, we computed the 4-Hz SNR of each sensor during the baseline test run. As anticipated, we found the highest SNR for occipital and parietal sensors (Figure 2A); we found a similar pattern of activity at 20 Hz (Figure 2B). Visual inspection of the topographies suggests the peak activity of the 20-Hz response was more occipital than the 4-Hz response. Based on these results and previous electrophysiological work on binocular disparity (Cottereau, McKee, & Norcia, 2012; Cottereau et al., 2011), we included all parietal and occipital (i.e., Oz, O1, O2, POz, PO3, PO4, PO7, PO8, Pz, P1, P2, P3, P4, P5, P6, P7, and P8) sensors in the main SSVEP analysis.

Figure 2. 

SSVEP in response to the test stimulus. (A–B) Topographic map showing the (A) 4 Hz and (B) 20 Hz SSVEP SNR in response to the test stimulus without prior adaptation. (C) Same as A, but for the difference in response to the test stimulus following adaptation to either correlated or aRDSs across. (D) The SSVEP SNR response spectra (averaged across parietal and occipital sensors) to the test stimulus following adaptation to cRDS/aRDS, for all participants (n = 31). The aRDS spectrum is horizontally offset to facilitate comparison with the cRDS spectra. (E) Same as C, but isolating the depth alternation (4 Hz) and dot refresh (20 Hz) frequencies. A repeated-measures ANOVA of SSVEP SNR revealed a main effect of frequency (4/20 Hz; F(1, 30) = 14.59, p = 6.2e−4) and an interaction between frequency and adaptation (cRDS/aRDS; F(1, 30) = 7.19, p = .012), but no main effect of adaptation (F(1, 30) = 2.76, p = .107). For participants who completed the baseline measurement (n = 15), F shows the same as D, but referenced to baseline SSVEP amplitude. We found numerical differences in the same direction between conditions at 8 and 12 Hz harmonics, but these did not reach thresholds for statistical significance (paired t test, 8 Hz: t(30) = 1.50, p = .144; 12 Hz: t(30) = 1.40, p = .171). Consistent with previous work showing that looming stimuli are more salient than receding stimuli (Franconeri & Simons, 2003), we found peaks (significantly above the level of noise) at the asymmetric harmonics (2 and 6 Hz) indicating a difference in response to the near and far stimuli (t test, 2 Hz: cRDS, t(30) = 7.09, p = 3.5e−8, aRDS, t(30) = 7.49, p = 1.2e−8; 6 Hz: cRDS, t(30) = 9.91, p = 4.1e−11, aRDS, t(30) = 6.34, p = 2.7e−7). However, we found no difference in the amplitude of these peaks between adaptation conditions (paired t test, 2 Hz: t(30) = 0.87, p = .389; 6 Hz: t(30) = 0.58, p = .564). (G) The average paired difference in 4 Hz SSVEP SNR between aRDS/cRDS conditions. Asterisks in A and B indicate sensors with SNR significantly than >1 following Bonferroni correction (N = 64, α = 7.8e−8); asterisks in E–G indicate significant differences. Error bars in E–F indicate SEM. Error bars in G indicate 95% confidence intervals.

Figure 2. 

SSVEP in response to the test stimulus. (A–B) Topographic map showing the (A) 4 Hz and (B) 20 Hz SSVEP SNR in response to the test stimulus without prior adaptation. (C) Same as A, but for the difference in response to the test stimulus following adaptation to either correlated or aRDSs across. (D) The SSVEP SNR response spectra (averaged across parietal and occipital sensors) to the test stimulus following adaptation to cRDS/aRDS, for all participants (n = 31). The aRDS spectrum is horizontally offset to facilitate comparison with the cRDS spectra. (E) Same as C, but isolating the depth alternation (4 Hz) and dot refresh (20 Hz) frequencies. A repeated-measures ANOVA of SSVEP SNR revealed a main effect of frequency (4/20 Hz; F(1, 30) = 14.59, p = 6.2e−4) and an interaction between frequency and adaptation (cRDS/aRDS; F(1, 30) = 7.19, p = .012), but no main effect of adaptation (F(1, 30) = 2.76, p = .107). For participants who completed the baseline measurement (n = 15), F shows the same as D, but referenced to baseline SSVEP amplitude. We found numerical differences in the same direction between conditions at 8 and 12 Hz harmonics, but these did not reach thresholds for statistical significance (paired t test, 8 Hz: t(30) = 1.50, p = .144; 12 Hz: t(30) = 1.40, p = .171). Consistent with previous work showing that looming stimuli are more salient than receding stimuli (Franconeri & Simons, 2003), we found peaks (significantly above the level of noise) at the asymmetric harmonics (2 and 6 Hz) indicating a difference in response to the near and far stimuli (t test, 2 Hz: cRDS, t(30) = 7.09, p = 3.5e−8, aRDS, t(30) = 7.49, p = 1.2e−8; 6 Hz: cRDS, t(30) = 9.91, p = 4.1e−11, aRDS, t(30) = 6.34, p = 2.7e−7). However, we found no difference in the amplitude of these peaks between adaptation conditions (paired t test, 2 Hz: t(30) = 0.87, p = .389; 6 Hz: t(30) = 0.58, p = .564). (G) The average paired difference in 4 Hz SSVEP SNR between aRDS/cRDS conditions. Asterisks in A and B indicate sensors with SNR significantly than >1 following Bonferroni correction (N = 64, α = 7.8e−8); asterisks in E–G indicate significant differences. Error bars in E–F indicate SEM. Error bars in G indicate 95% confidence intervals.

Overall, we observed differences in activity in response to the test following adaptation to cRDS and aRDS (Figure 2C, D). To test the prediction that prolonged viewing of binocular mismatches increases neural excitability, we computed the average SSVEP SNR across parietal and occipital sensors and compared responses at the depth change frequency (4 Hz). In line with the prediction, we found that SSVEP SNR was significantly higher following adaptation to aRDS compared with cRDS (paired t test, all participants: t(30) = 2.49, p=.019; participants with baseline: t(21) = 2.77, p = .011; Figure 2E; 19 of 31 participants showed the effect). A possible concern is that the difference in SSVEP SNR between adaptation conditions was due to a decrease in excitability following adaptation to cRDS, rather than an increase following adaptation to aRDS. However, we found no evidence for this: Whereas adaptation to aRDS significantly increased SSVEP SNR relative to baseline (paired t test, t(21) = 2.39, p = .026), adaptation to cRDS produced no significant change (paired t test, t(21) = 0.74, p = .469; Figure 2F); these results are consistent with the interpretation that adaptation to binocular mismatches increased neural excitability.

Another possible concern is that the increased excitability we observed in the primary visual cortex following adaptation to aRDS is not specific to stereopsis, but a generalized effect. However, we found no evidence for a difference in SSVEP amplitude at the frequency of the dot refresh (20 Hz) between aRDS and cRDS adaptation conditions (paired t test, t(30) = 0.60, p = .552; Figure 2E), suggesting that the effect relates to changes in depth and not luminance. Another possible concern is that the effect might be driven by a subset of participants with low SSVEP amplitude across all conditions. Specifically, SSVEP amplitude measurements are less reliable at low values; thus, a subset of unreliable values could potentially yield a false positive. However, although we found significant correlations between baseline, cRDS, and aRDS SSVEP amplitude (Pearson correlation: participants with baseline: n = 22, baseline–cRDS: r = .489, p = .021; baseline–aRDS: r = .596, p = .003; cRDS–aRDS: r = .897, p = 1.5e−13; all participants, n = 31: cRDS–aRDS: r = .92, p = 1.5e−13; Figure 3AC), we found no evidence of a relationship between baseline amplitude and the difference between aRDS and cRDS (Pearson correlation, n = 22, r = .219, p = .327; Figure 3D).

Figure 3. 

Relationship between SSVEP SNR across conditions. (A) SSVEP SNR measured in the response to the test stimulus following cRDS adaptation as a function of SSVEP SNR without adaptation. (B) Same as A, but following aRDS adaptation. (C) Same as B, but as a function of SSVEP SNR following cRDS adaptation. Black and gray dots indicate participants who did and did not complete the baseline condition, respectively. Black and gray lines indicate the least squares regression not including (n = 22) and including (n = 33) participants who did not complete the baseline, respectively. The red dotted lines in A–C indicate the first diagonal (y = x). (D) The difference in the effect of cRDS and aRDS adaptation on SSVEP SNR as a function of SSVEP SNR without adaptation.

Figure 3. 

Relationship between SSVEP SNR across conditions. (A) SSVEP SNR measured in the response to the test stimulus following cRDS adaptation as a function of SSVEP SNR without adaptation. (B) Same as A, but following aRDS adaptation. (C) Same as B, but as a function of SSVEP SNR following cRDS adaptation. Black and gray dots indicate participants who did and did not complete the baseline condition, respectively. Black and gray lines indicate the least squares regression not including (n = 22) and including (n = 33) participants who did not complete the baseline, respectively. The red dotted lines in A–C indicate the first diagonal (y = x). (D) The difference in the effect of cRDS and aRDS adaptation on SSVEP SNR as a function of SSVEP SNR without adaptation.

Finally, another possible explanation for the effect might be that observers' attentional allocation during the adaptation period varied between conditions. However, we found no evidence for a difference in performance on the attentionally demanding Vernier task between conditions (paired t test, t(30) = 1.22, p = .23). Similarly, we found no evidence for a difference in eye movements in response to either the adaptation or test stimuli (Figure 4A, B).

Figure 4. 

Comparison of eye movements between (cRDS/aRDS) adaptation conditions. We assessed whether the adaptation effects observed could be explained by differences in eye movements during stimulus presentation by comparing observers' eye position during the adaptation and test periods between cRDS and aRDS adaptation conditions. (A) Average vergence and version eye movements for near and far stimuli presented during the cRDS and aRDS adaptation periods. (B) Same as A, but for test periods. The dashed lines in A indicate stimulus onset and offset; color-shaded regions indicate ±1 SEM.

Figure 4. 

Comparison of eye movements between (cRDS/aRDS) adaptation conditions. We assessed whether the adaptation effects observed could be explained by differences in eye movements during stimulus presentation by comparing observers' eye position during the adaptation and test periods between cRDS and aRDS adaptation conditions. (A) Average vergence and version eye movements for near and far stimuli presented during the cRDS and aRDS adaptation periods. (B) Same as A, but for test periods. The dashed lines in A indicate stimulus onset and offset; color-shaded regions indicate ±1 SEM.

Although the strongest response to the 4-Hz depth alternation of the test stimulus in the baseline condition was in posterior sensors (Figure 2A), there was a region of elevated SNR in anterior frontal electrodes. This activity was also reflected in the topography of condition differences (Figure 2C). Eye-tracking results show this is unlikely to be due to differences in eye movements. Rather, consistent with previous work (Cottereau et al., 2011), this activity likely reflects the distributed involvement of multiple neural structures in the processing of binocular disparity.

ERP Analysis of Adaptation Stimuli

The primary SSVEP analysis revealed a difference in stereoscopic event-related neural responsiveness following adaptation to aRDS but not cRDS, relative to baseline, consistent with the prediction that adaptation to binocular mismatches increases neural excitability. If the cRDS/aRDS adaptation stimuli have different functional consequences, this indicates that these stimuli evoked different patterns of activity during adaptation. Although the central aim of the experiment was to test the consequences of adaptation to cRDS/aRDS, the ERPs evoked by these stimuli during the adaptation period may inform the mechanism by which different effects of adaptation were produced. Thus, in an exploratory analysis, we computed the difference between cRDS/aRDS ERPs (averaged across all presentations, across all sensors). We found that the sensors that showed the greatest difference were regionally similar to those used in the SSVEP analysis (Figure 5A). Given the topographic similarity of differences in activity evoked by test and adaptor stimuli between conditions, we then compared the pooled activity of adaptor ERPs from the same (occipital and parietal) sensors used in the previous analysis. We found ERP amplitude at time points consistent with the P1 component were significantly smaller in response to aRDS compared with cRDS, but the amplitude of those consistent with the N1 component were significantly larger (paired t test, peak t(30) = 3.54, p = .001; Figure 5B).

Figure 5. 

ERPs in response to the adaptation stimuli. (A) Topographic maps showing average neural activity in response to the (cRDS/aRDS) adaptation stimuli and the difference between them at 100, 200, 300, and 400 msec after stimulus onset. (B) Neural activity in response to the (cRDS/aRDS) adaptation stimuli, averaged over the occipital and parietal sensors, as a function of time. The dashed line indicates stimulus onset; color-shaded regions indicate ±1 SEM, and gray-shaded bar indicates significant differences between conditions.

Figure 5. 

ERPs in response to the adaptation stimuli. (A) Topographic maps showing average neural activity in response to the (cRDS/aRDS) adaptation stimuli and the difference between them at 100, 200, 300, and 400 msec after stimulus onset. (B) Neural activity in response to the (cRDS/aRDS) adaptation stimuli, averaged over the occipital and parietal sensors, as a function of time. The dashed line indicates stimulus onset; color-shaded regions indicate ±1 SEM, and gray-shaded bar indicates significant differences between conditions.

Anticorrelated Test Stimulus

We used a cRDS test stimulus, instead of an aRDS, to measure changes in neural excitability following adaptation to anticorrelated images as cRDS evoke a larger response than aRDS (Petrig et al., 1981; Braddick et al., 1980), providing a better SNR and thus a more sensitive measure of neural change. However, a possible concern is that the reduction in SSVEP amplitude we observed in response to the change in the depth of a cRDS following adaptation to anticorrelation was influenced by the difference between adaptation and test stimuli. That is, the cRDS test stimulus evoked a reduced neural response following adaptation to aRDS compared with cRDS because the difference between adaptation and test stimulus was more pronounced in the aRDS condition. To test this possibility, we repeated the experiment, on a new cohort of participants, using an aRDS test stimulus (Figure 6A).

Figure 6. 

Stimuli and results from the control experiment. (A) Example of the aRDS test stimuli used in the control experiment. (B)The SSVEP SNR response spectra (averaged across parietal and occipital sensors) to the aRDS test stimulus following adaptation to cRDS/aRDS, for all participants (n = 22). The aRDS spectrum is horizontally offset to facilitate comparison with the cRDS spectra.

Figure 6. 

Stimuli and results from the control experiment. (A) Example of the aRDS test stimuli used in the control experiment. (B)The SSVEP SNR response spectra (averaged across parietal and occipital sensors) to the aRDS test stimulus following adaptation to cRDS/aRDS, for all participants (n = 22). The aRDS spectrum is horizontally offset to facilitate comparison with the cRDS spectra.

We included data from the same sensors used in the SSVEP analysis of the main experiment (i.e., Oz, O1, O2, POz, PO3, PO4, PO7, PO8, Pz, P1, P2, P3, P4, P5, P6, P7, and P8). The SSVEP SNR response spectra for adaptation conditions are shown in Figure 5B. Although there is a clear peak at the dot refresh frequency (20 Hz), there is little discernible peak at the depth alternation frequency (4 Hz); contrast this with the spectra from the main experiment where a peaks can be clearly resolved at 4 Hz and its harmonics (Figure 2D). Indeed, a comparison of the SSVEP SNR at 4 Hz between the baseline condition here and in the main experiment show that the amplitude was significantly lower here, in response to an aRDS test stimulus (independent t test, t(43) = 6.51, p = 7.3e−8). We found no significant difference in the 4-Hz SSVEP SNR between aRDS and cRDS adaptation conditions (paired t test, t(21) = 1.46, p = .159). However, if there were differential effects of adaptation to cRDS and aRDS on the potential evoked by the aRDS test stimulus, given the low amplitude of the signal, it is unlikely that this difference could be detected.

Although we did not find a reliable signal in response to the depth alternation of the anticorrelated test stimulus, we were able to reliably measure the ERP in response to the anticorrelated stimulus during the adaptation phase. This is likely because in the adaptation phase, the depth alternation was concurrent with the stimulus presentation; thus, the ERP reflected the response to both the change in depth and the stimulus onset. By contrast, in the test phase, the stimulus was presented constantly whereas depth alternated at 4 Hz; thus, the signal at 4 Hz reflects the response to depth alternation in isolation. These results suggest that the ERP measured in the adaptation phase (at least for the anticorrelation condition) was primarily a response to stimulus onset.

DISCUSSION

Electrophysiological recordings from macaque visual cortex (Janssen et al., 2003; Tsao et al., 2003; Cumming & Parker, 1997) and psychophysical work with humans (Katyal et al., 2018; Kingdom et al., 2018) has revealed the existence of cortical mechanisms tuned to mismatched features between the left and right eyes. Although the evidence supporting the existence of neurons tuned to mismatches is extensive, our understanding of their role in binocular vision remains limited. Here, we provide evidence that neural activity in the visual cortex may facilitate binocular vision through inhibition. In particular, we show that adaptation to mismatched binocular stimuli, that is, aRDS, produces increased excitability in the visual cortex in response to changes in depth.

Prolonged/repeated exposure typically produces a reduction in the responsiveness of stimulated neurons, that is, adaptation. Thus, one might expect adaptation to aRDS to reduce the net responsiveness of the visual cortex in response to a change in depth. Alternatively, one might anticipate that adaptation to aRDS to have no effect on the response to a cRDS, due to the perceptual dissimilarity of these stimuli, that is, aRDS do not produce a percept of depth. However, in contrast to these intuitive hypotheses, we found that adaptation to aRDS yields an increase in excitability. Although these results may seem surprising, they are consistent with the notion that neurons tuned to binocular mismatches can facilitate stereopsis by suppressing unlikely perceptual interpretations (Goncalves & Welchman, 2017).

The results of the SSVEP analysis of activity evoked by cRDS stimuli in the test period suggested that prolonged viewing of cRDS and aRDS resulted in adaptation of different neural ensembles or similar neural ensembles adapted to different extents. Analysis of the ERPs evoked by the adaptor stimuli confirmed this, revealing different patterns of activity corresponding to cRDS and aRDS stimuli. Specifically, we found differences between the P1 and N1 components. P1 and N1 components are thought to represent activity relating to early sensory processes; thus, the differences in amplitude of these components in response to aRDS and cRDS may reflect the differences in the engagement of excitatory vs. inhibitory mechanisms. The amplitude of N1 has been shown to increase with attentional allocation (Hillyard & Anllo-Vento, 2002; Polich, 1986; Van Voorhis & Hillyard, 1977; Haider, Spong, & Lindsley, 1964). As the amplitude of the N1 component was higher for aRDS, this may signal that observers allocated more attention to the aRDS than the cRDS, for example, because of its perceptual peculiarity. However, the results from the concurrent (attentionally demanding) task suggest otherwise: No differences in accuracy or response time were observed between conditions. Furthermore, no difference in amplitude between adaptation conditions was found for the P1 component, which is also modulated by attention (Hillyard & Anllo-Vento, 2002; Heinze et al., 1994).

A previous study found a reduction in SSVEP amplitude in response to a drifting sinusoidal grating, following adaptation to a grating drifting in the same direction (Ales & Norcia, 2009). It seems reasonable to expect that the response to the cRDS test stimulus would be reduced following adaptation to cRDS. Given that we did not find a reduction in excitability following adaptation to cRDS, this may indicate that adaptation to motion has a more pronounced neural signature than adaptation to binocular disparity. However, there are other methodological differences that may also account for this apparent discrepancy. The stimuli used in our experiment covered a visual region approximately 20% the size of that used by Ales and Norcia (2009), thereby eliciting adaptation of fewer neurons. Furthermore, we adapted participants to and tested two disparities (±10 arcmin) within the same block, whereas Ales and Norcia (2009) separately adapted to different directions within each block. In designing the experiment, we predicted that we would find the greatest overall effect on excitability by adapting both crossed and uncrossed disparities. However, our results may indicate that the effects of adaptation are more readily detected using the SSVEP method when one population of neurons is adapted at a time. Another possible explanation is that the correlation between left- and right-eye images in the cRDS is more similar to that observed in the natural environment, compared with the aRDS, which has an artificially low correlation. Thus, the tolerance for cRDS may be higher than that for aRDS, and effects of adaptation may be subtler. However, we found a reduction in the amplitude of the ERP over the course of the adaptation period for both aRDS and cRDS, indicating that adaptation had a measurable effect on neural activity for both types of stimuli.

A possible concern is that the increase in SSVEP amplitude following adaptation to aRDS, compared with cRDS, was due to the aRDS stimulus being more perceptually dissimilar to the test stimulus than the cRDS. In particular, the increased response may be due to less expectation of the test stimulus in the aRDS condition. The results from the control experiment, in which we used an aRDS test stimulus, provided inconclusive evidence for this possibility, as the amplitude of the SSVEP SNR was too low to produce reliable estimates. This is consistent with previous work showing cRDS evoke a larger response than aRDS (Petrig et al., 1981; Braddick et al., 1980) and may suggest that the response to the cRDS probe in the main experiment primarily originated from areas where the correspondence problem had been solved. However, in the baseline condition of the main experiment, the test stimulus was preceded by a period in which a gray background was presented; this is arguably more dissimilar from the test stimulus than either cRDS/aRDS adaptor. Thus, if SSVEP amplitude reflected the similarity between the test stimuli and preceding images, we would expect SSVEP amplitude to be highest in the baseline condition where the dissimilarity was highest. However, we found no evidence for this: baseline amplitude was significantly lower than that following adaptation to aRDS.

Although EEG has relatively high temporal resolution, the spatial resolution of the technique is limited. Thus, a limitation of the current study is that we cannot make precise statements about the likely neural locus of adaptation to binocular anticorrelation in the visual cortex. fMRI is known to have much better spatial resolution compared with EEG; however, excitatory and inhibitory activity cannot be differentiated from BOLD signal, restricting the diagnostic utility of this technique in establishing the role of “what not” mechanisms.

It is interesting to consider the behavioral implications of our results. Previous work has shown that adaptation to binocular anticorrelation (also known as interocular contrast differences) reduce observers' sensitivity to binocular anticorrelation (Katyal et al., 2018; Kingdom et al., 2018). We did not find a difference in performance on the irrelevant task used to maintain observers' eye movements and attention; however, it is possible that the increased neural excitability resulting from adaptation to binocular anticorrelation may have altered performance on a task involving depth judgments. Future work could test whether reducing the sensitivity of neural systems tuned to binocular anticorrelation through adaptation increases sensitivity to binocular correlation, for example, by reducing neural inhibition.

The current results have implications beyond stereopsis. There is theoretical and empirical evidence supporting the existence of neurons tuned to mismatches from studies of stereopsis (Tsao et al., 2003; Prince, Cumming, & Parker, 2002; DeAngelis, Ohzawa, & Freeman, 1991), binocular rivalry (Katyal et al., 2018; Kingdom et al., 2018; Said & Heeger, 2013), and integration of cues within (Rideaux & Welchman, 2018; Kim, Angelaki, & Deangelis, 2015; Nadler et al., 2013) and between sensory modalities (Kim, Pitkow, Angelaki, & DeAngelis, 2016; Gu, Angelaki, & DeAngelis, 2008; Morgan, DeAngelis, & Angelaki, 2008). Here, we provide evidence suggesting a role for mechanisms tuned to mismatches that may facilitate inference by driving suppression.

Acknowledgments

We thank N. Goncalves for detailed discussions and comments on the article. This work was supported by the Leverhulme Trust (ECF-2017-573 to R. R.), the Isaac Newton Trust (17.08(o) to R. R.), and the Wellcome Trust (095183/Z/10/Z to A. E. W. and 206495/Z/17/Z to E. M.).

Reprint requests should be sent to Reuben Rideaux, Department of Psychology, University of Cambridge, CB2 3EB, United Kingdom, or via e-mail: reuben.rideaux@gmail.com.

REFERENCES

REFERENCES
Ales
,
J. M.
, &
Norcia
,
A. M.
(
2009
).
Assessing direction-specific adaptation using the steady-state visual evoked potential: Results from EEG source imaging
.
Journal of Vision
,
9
,
8
.
Braddick
,
O.
,
Atkinson
,
J.
,
Julesz
,
B.
,
Kropfl
,
W.
,
Bodis-Wollner
,
I.
, &
Raab
,
E.
(
1980
).
Cortical binocularity in infants
.
Nature
,
288
,
363
365
.
Brainard
,
D. H.
(
1997
).
The psychophysics toolbox
.
Spatial Vision
,
10
,
433
436
.
Cornelissen
,
F. W.
,
Peters
,
E. M.
, &
Palmer
,
J.
(
2002
).
The Eyelink toolbox: Eye tracking with MATLAB and the Psychophysics toolbox
.
Behavior Research Methods, Instruments & Computers
,
34
,
613
617
.
Cottereau
,
B. R.
,
McKee
,
S. P.
,
Ales
,
J. M.
, &
Norcia
,
A. M.
(
2011
).
Disparity-tuned population responses from human visual cortex
.
Journal of Neuroscience
,
31
,
954
965
.
Cottereau
,
B. R.
,
McKee
,
S. P.
,
Ales
,
J. M.
, &
Norcia
,
A. M.
(
2012
).
Disparity-Specific Spatial Interactions: Evidence from EEG Source Imaging
.
Journal of Neuroscience
,
32
,
826
840
.
Cottereau
,
B. R.
,
McKee
,
S. P.
, &
Norcia
,
A. M.
(
2012
).
Bridging the gap: Global disparity processing in the human visual cortex
.
Journal of Neurophysiology
,
107
,
2421
2429
.
Cumming
,
B. G.
, &
Parker
,
A. J.
(
1997
).
Responses of primary visual cortical neurons to binocular disparity without depth perception
.
Nature
,
389
,
280
283
.
DeAngelis
,
G. C.
,
Ohzawa
,
I.
, &
Freeman
,
R. D.
(
1991
).
Depth is encoded in the visual cortex by a specialized receptive field structure
.
Nature
,
352
,
156
159
.
Delorme
,
A.
, &
Makeig
,
S.
(
2004
).
EEGLAB: An open source toolbox for analysis of single-trial EEG dynamics including independent component analysis
.
Journal of Neuroscience Methods
,
134
,
9
21
.
Fleet
,
D. J.
,
Wagner
,
H.
, &
Heeger
,
D. J.
(
1996
).
Neural encoding of binocular disparity: Energy models, positionshifts and phase shifts
.
Vision Research
,
36
,
1839
1857
.
Franconeri
,
S. L.
, &
Simons
,
D. J.
(
2003
).
Moving and looming stimuli capture attention
.
Perception & Psychophysics
,
65
,
999
1010
.
Goncalves
,
N. R.
, &
Welchman
,
A. E.
(
2017
).
“What not” detectors help the brain see in depth
.
Current Biology
,
27
,
1403
1412
.
Gu
,
Y.
,
Angelaki
,
D. E.
, &
DeAngelis
,
G. C.
(
2008
).
Neural correlates of multisensory cue integration in macaque MSTd
.
Nature Neuroscience
,
11
,
1201
1210
.
Haider
,
M.
,
Spong
,
P.
, &
Lindsley
,
D. B.
(
1964
).
Attention, vigilance, and cortical evoked-potentials in humans
.
Science
,
145
,
180
182
.
Heinze
,
H. J.
,
Mangun
,
G. R.
,
Burchert
,
W.
,
Hinrichs
,
H.
,
Scholz
,
M.
,
Münte
,
T. F.
, et al
(
1994
).
Combined spatial and temporal imaging of brain activity during visual selective attention in humans
.
Nature
,
372
,
543
546
.
Hillyard
,
S. A.
, &
Anllo-Vento
,
L.
(
2002
).
Event-related brain potentials in the study of visual selective attention
.
Proceedings of the National Academy of Sciences, U.S.A.
,
95
,
781
787
.
Janssen
,
P.
,
Vogels
,
R.
,
Liu
,
Y.
, &
Orban
,
G. A.
(
2003
).
At least at the level of inferior temporal cortex, the stereo correspondence problem is solved
.
Neuron
,
37
,
693
701
.
Julesz
,
B.
, &
Chang
,
J. J.
(
1976
).
Interaction between pools of binocular disparity detectors tuned to different disparities
.
Biological Cybernetics
,
22
,
107
119
.
Katyal
,
S.
,
Vergeer
,
M.
,
He
,
S.
,
He
,
B.
, &
Engel
,
S. A.
(
2018
).
Conflict-sensitive neurons gate interocular suppression in human visual cortex
.
Scientific Reports
,
8
,
1239
.
Kim
,
H. R.
,
Angelaki
,
D. E.
, &
Deangelis
,
G. C.
(
2015
).
A novel role for visual perspective cues in the neural computation of depth
.
Nature Neuroscience
,
18
,
129
137
.
Kim
,
H. R.
,
Pitkow
,
X.
,
Angelaki
,
D. E.
, &
DeAngelis
,
G. C.
(
2016
).
A simple approach to ignoring irrelevant variables by population decoding based on multisensory neurons
.
Journal of Neurophysiology
,
116
,
1449
1467
.
Kingdom
,
F. A. A.
,
Jennings
,
B. J.
, &
Georgeson
,
M. A.
(
2018
).
Adaptation to interocular difference
.
Journal of Vision
,
18
,
9
.
Marr
,
D.
, &
Poggio
,
T.
(
1976
).
Cooperative computation of stereo disparity
.
Science
,
194
,
283
287
.
Morgan
,
M. L.
,
DeAngelis
,
G. C.
, &
Angelaki
,
D. E.
(
2008
).
Multisensory integration in macaque visual cortex depends on cue reliability
.
Neuron
,
59
,
662
673
.
Nadler
,
J. W.
,
Barbash
,
D.
,
Kim
,
H. R. R.
,
Shimpi
,
S.
,
Angelaki
,
D. E.
, &
DeAngelis
,
G. C.
(
2013
).
Joint representation of depth from motion parallax and binocular disparity cues in macaque area MT
.
Journal of Neuroscience
,
33
,
14061
14074
.
Ohzawa
,
I.
,
DeAngelis
,
G.
, &
Freeman
,
R.
(
1990
).
Stereoscopic depth discrimination in the visual cortex: Neurons ideally suited as disparity detectors
.
Science
,
249
,
1037
1041
.
Pelli
,
D. G.
(
1997
).
The Videotoolbox software for visual psychophysics: Transforming numbers into movies
.
Spatial Vision
,
10
,
437
442
.
Petrig
,
B.
,
Julesz
,
B.
,
Kropfl
,
W.
,
Baumgartner
,
G.
, &
Anliker
,
M.
(
1981
).
Development of stereopsis and cortical binocularity in human infants: Electrophysiological evidence
.
Science
,
213
,
1402
1405
.
Polich
,
J.
(
1986
).
Attention, probability, and task demands as determinants of P300 latency from auditory stimuli
.
Electroencephalography and Clinical Neurophysiology
,
63
,
251
259
.
Preston
,
T. J.
,
Li
,
S.
,
Kourtzi
,
Z.
, &
Welchman
,
A. E.
(
2008
).
Multivoxel pattern selectivity for perceptually relevant binocular disparities in the human brain
.
Journal of Neuroscience
,
28
,
11315
11327
.
Prince
,
S. J.
,
Cumming
,
B. G.
, &
Parker
,
A. J.
(
2002
).
Range and mechanism of encoding of horizontal disparity in macaque V1
.
Journal of Neurophysiology
,
87
,
209
221
.
Read
,
J. C.
, &
Cumming
,
B. G.
(
2007
).
Sensors for impossible stimuli may solve the stereo correspondence problem
.
Nature Neuroscience
,
10
,
1322
1328
.
Rideaux
,
R.
, &
Welchman
,
A. E.
(
2018
).
Proscription supports robust perceptual integration by suppression in human visual cortex
.
Nature Communications
,
9
,
1502
.
Said
,
C. P.
, &
Heeger
,
D. J.
(
2013
).
A model of binocular rivalry and cross-orientation suppression
.
PLOS Computational Biology
,
9
,
e1002991
.
Tsao
,
D. Y.
,
Conway
,
B. R.
, &
Livingstone
,
M. S.
(
2003
).
Receptive fields of disparity-tuned simple cells in macaque V1
.
Neuron
,
38
,
103
114
.
Van Voorhis
,
S.
, &
Hillyard
,
S. A.
(
1977
).
Visual evoked potentials and selective attention to points in space
.
Perception & Psychophysics
,
22
,
54
62
.