Although shape perception is primarily considered a function of the ventral visual pathway, previous research has shown that both dorsal and ventral pathways represent shape information. Here, we examine whether the shape-selective electrophysiological signals observed in dorsal cortex are a product of the connectivity to ventral cortex or are independently computed. We conducted multiple EEG studies in which we manipulated the input parameters of the stimuli so as to bias processing to either the dorsal or ventral visual pathway. Participants viewed displays of common objects with shape information parametrically degraded across five levels. We measured shape sensitivity by regressing the amplitude of the evoked signal against the degree of stimulus scrambling. Experiment 1, which included grayscale versions of the stimuli, served as a benchmark establishing the temporal pattern of shape processing during typical object perception. These stimuli evoked broad and sustained patterns of shape sensitivity beginning as early as 50 msec after stimulus onset. In Experiments 2 and 3, we calibrated the stimuli such that visual information was delivered primarily through parvocellular inputs, which mainly project to the ventral pathway, or through koniocellular inputs, which mainly project to the dorsal pathway. In the second and third experiments, shape sensitivity was observed, but in distinct spatio-temporal configurations from each other and from that elicited by grayscale inputs. Of particular interest, in the koniocellular condition, shape selectivity emerged earlier than in the parvocellular condition. These findings support the conclusion of distinct dorsal pathway computations of object shape, independent from the ventral pathway.
A prominent view of the cortical visual system holds that there are two distinct pathways, one ventral and one dorsal, that support perception and visuomotor control, respectively (Goodale & Milner, 1992). Growing evidence has challenged this binary segregation, revealing, for example, that object perception is not under the sole purview of the ventral pathway but, rather, is also supported by representations derived by the dorsal pathway (Freud, Culham, Plaut, & Behrmann, 2017; Bracci & Op de Beeck, 2016; Konen & Kastner, 2008; for recent reviews, see Freud, Plaut, & Behrmann, 2016; Xu, 2018).
In the last decade, there has been an increasing number of studies that have documented sensitivity to shape in parietal cortex in nonhuman primates and in humans (Freud, Culham, et al., 2017; Bracci & Op de Beeck, 2016; Van Dromme, Premereur, Verhoef, Vanduffel, & Janssen, 2016; Zachariou, Klatzky, & Behrmann, 2014; Xu, 2018; Konen & Kastner, 2008). In one such study, using fMRI, Freud, Culham, et al. (2017) showed that shape perception—a critical component of object recognition—is subserved by both the ventral and dorsal visual pathways. Specifically, this study used images of common objects as inputs and increasingly scrambled the shape of the objects across five levels (see example in Figure 1). By regressing the level of scrambling against the beta weights, an index of shape sensitivity was derived for each voxel. Interestingly, the fMRI results indicated that shape sensitivity increased along a gradient from more posterior to more anterior regions and that this was true for both the ventral and dorsal visual pathways. Furthermore, using representational similarity analysis, they showed that activity in some regions of the two pathways, namely, the posterior part of the dorsal pathway including V3a, IPS0, and IPS1, was highly correlated with regions of the lateral ventral pathway, including LO1, LO2, and TO1. These same regions in both pathways were also correlated with recognition performance, suggesting that both pathways likely contribute to perception.
A key question that remains to be addressed is whether the shape representations documented in dorsal cortex are the result of a cascade of information from ventral cortex or whether they are independently computed. Xu (2018), for example, proposes that the visual shape representations in posterior parietal cortex are the output from occipito-temporal cortex and that the dorsal regions “upload” the information from ventral cortex depending on task demands and the observer's behavioral goals. In contrast, Freud et al. (2016) and Freud, Ganel, et al. (2017) argued that the shape information in each of the two pathways was at least partially independently computed. Evidence for this claim comes from the finding that patients with lesions of ventral agnosia still evinced a normal BOLD profile of shape representation in dorsal cortex and that, even when any residual activation from right ventral cortex was covaried from the dorsal signal, the dorsal shape sensitivity was still evident (Freud, Ganel, et al., 2017). Furthermore, although one of the patients had widespread ventral damage bilaterally, sensitivity to the properties of object shapes was still observed in a series of behavioral studies tapping object perception. Although this evidence clearly supports a dissociation in individuals with cortical injury, there is still a need to establish the same pattern in neurotypical individuals. One approach that might offer support for possible independence of the pathways is to demonstrate that the time course of activation of shape representations differs in the two pathways. This approach forms the primary focus of the current article.1 The primary motivation for examination of the temporal dynamics of shape processing, currently via EEG, is that dorsal and ventral shape representations may emerge at different times, and therefore, one can examine the extent to which they are distinct in both of the other experiments.
Temporal Dynamics of Shape Processing
There are a number of studies that have utilized EEG to explore the temporal dynamics of shape processing in visual perception, although these studies do not examine shape perception per se. For example, shape has been used in conjunction with color to explore the time course of selective visual attention (Proverbio, Burco, del Zotto, & Zani, 2004; Smid, Jakob, & Heinze, 1999) or as a control for more complex visual stimuli (George, Jemel, Fiori, & Renault, 1997). As another example, Lucan, Foxe, Gomez-Ramirez, Sathian, and Molholm (2010) characterized the time course of tactile shape stimuli to show differences between a shape discrimination task and a shape duration task, but again, the major focus was not on the neural basis of shape perception per se. There have been many studies that have explored the temporal dynamics of shape processing more directly, but these investigations largely focus on specific temporal components, for example, the N170 and N250 components (Tanaka, Curran, Porterfield, & Collins, 2006; Eimer, 2000; Bentin, Allison, Puce, Perez, & McCarthy, 1996) or spectral components, as in the context of recent fast periodic visual stimulation paradigms (Rossion, 2014), and almost all of these studies explore face processing rather than object perception more generally. Here, we elucidate the temporal profile of the cortical response to common objects with specific attention to the similarity or differences of the signature in the dorsal and ventral pathways.
Evaluating the differential contribution of dorsal and ventral cortices to shape perception is not without its challenges. For example, given the dense structural, and functional, connectivity between parietal and temporal cortices, it has proven difficult to ascribe processing to one or the other pathway independently, particularly in the context of a slow BOLD signal (Takemura et al., 2015; Yeatman, Dougherty, Ben-Shachar, & Wandell, 2012). This is also difficult in the context of EEG given that scalp recordings do not easily permit veridical source localization of signals. To circumvent these limitations, the approach we adopt is one in which we manipulate the input properties of the stimuli so as to bias processing to one pathway or the other. In particular, we manipulated the input parameters of the stimuli so that information was propagated primarily through one of two thalamocortical pathways: parvocellular or koniocellular.
The parvocellular pathway projects almost exclusively to ventral stream structures (Merigan & Maunsell, 1993) as its constituent midget ganglion cells are color sensitive and responsive to red-green isoluminant stimuli (Livingstone & Hubel, 1988). Koniocellular cells, on the other hand, project to dorsal pathway areas (Casagrande, 1994), by virtue of the bistratified ganglion cells and intralaminar cells (Das & Huxlin, 2010). The logic is that if there are differences between the evoked responses when information is delivered primarily to each of the two pathways, then the differences must reflect, at a minimum, the cortical contribution of each pathway. Moreover, because there is rapid propagation of signals between the two pathways, the earlier segments of the time course (before widespread cascade) may be particularly informative.
Importantly, visual information is also projected via a third thalamocortical visual tract, the magnocellular pathway, preferentially to the dorsal pathway, but also to the ventral pathway. The magnocellular pathway carries achromatic, low-resolution information to cortex (Merigan & Maunsell, 1993). Here, we chose to leverage the elegant approach of a recent study (Almeida, Fintzi, & Mahon, 2013) that used parvocellular and koniocellular manipulations to separate contributions of the visual streams. Furthermore, this approach was most appropriate for our established stimuli and permitted behavioral benchmarking in a similar fashion to our fMRI study (Freud et al., 2017), without manipulating spatial frequency of the stimuli.
The approach of biasing processing in this manner has already generated insight into the unique contributions of ventral and dorsal processing (Kveraga, Boshyan, & Bar, 2007). For example, Almeida et al. (2013) examined the role of the two visual pathways in the processing of tools by exploring the asymmetry in BOLD signals when visual information was propagated to either the parvocellular and koniocellular channels. In the current study, we utilize the same logic to elucidate the temporal dynamics of shape processing and its relative dependencies on ventral and dorsal pathways by biasing the input to one or the other pathway and measuring the ensuing electrophysiological response.
The Current Study
We conducted three separate EEG studies to explore the temporal dynamics of shape processing and its relative dependencies on each of the pathways when the processing of inputs is biased one way or the other. In Experiment 1, we adopted the same stimulus scrambling paradigm used previously (Freud, Culham, et al., 2017; Lerner, Hendler, Ben-Bashat, Harel, & Malach, 2001; Figure 1A) to establish the feasibility of this paradigm in the EEG environment. That is, Experiment 1 is a proof of concept that we use to ensure that we can evoke a gradation of responses similar to those found by Freud, Culham, et al. (2017) using fMRI. By adopting the stimuli used previously, our first experiment extends the findings of a previous report and establishes the temporal pattern of shape sensitivity. Unsurprisingly, given the cortical response to grayscale stimuli, the EEG waveforms reveal the full and joint contributions of both pathways but cannot adjudicate the question of independence of signals. In Experiments 2 and 3, we altered the properties of these same object stimuli to bias processing to the ventral or dorsal pathways, respectively. In Experiment 2, isoluminant red-green stimuli were constructed so as to engage the parvocellular system, which synapses in Layer 4 of V1 and sends information primarily to the ventral pathway (Merigan & Maunsell, 1993; see Almeida et al., 2013; Kveraga et al., 2007; Freud, Robinson, & Behrmann, 2018, for a similar approach). In Experiment 3, stimuli were designed to bias processing to the koniocellular system, which synapses in both Layer 1 of V1 and directly in V5/MT of the dorsal pathway (Dobkins, 2000; Hendry & Reid, 2000; Casagrande, 1994). Almeida et al. (2013) were the first to document a role of the koniocellular pathway in high-level object processing. One previous EEG study documented the role of tritanopic stimuli in rapid processing of motion in humans and suggested that the dominant input to cortex in the koniocellular pathway is directly to V5/MT (Morand et al., 2000). Thus, the waveforms evoked by tritanopic stimuli likely reflect information primarily computed, at least initially, in the dorsal pathway.
We hypothesized that, if the ventral and dorsal shape representations were independent, biasing input separately to each cortical visual stream would elicit distinct spatio-temporal signatures of shape perception. Moreover, given the rapid transmission of signals to the dorsal pathway (Bar et al., 2006), we might expect to see an earlier onset in Experiment 3 (koniocellular) than in Experiment 2 (parvocellular; see also Morand et al., 2000). Such a result would be inconsistent with the claim that dorsal cortex is merely the recipient of activation from ventral cortex as, in this case, activation of the former would precede the latter. In addition, a focus on the earliest time points at which shape sensitivity emerges would permit an especially informative comparison between parvocellular- and koniocellular-driven processes as there is less “contamination” of information across pathways, relative to later time points in the poststimulus epoch. Critically, any comparisons specifically aimed to dissociate dorsal and ventral contributions to shape processing must be made between Experiments 2 and 3.
Sixty-two right-handed individuals completed the EEG procedure (35 women; mean age = 20.74 years, SD = 3.41). Three participants were excluded from the analysis because of excessive artifacts during the EEG recording, resulting in 20 participants in each of the first two experiments and 19 in the final experiment. An additional 32 naive participants, who did not complete the EEG procedure, completed a separate stimulus validation experiment to provide recognition rates and validation of stimuli for use in Experiments 2 and 3. All participants were paid or received class credit for their participation. All participants reported having normal or corrected-to-normal vision and normal color vision and did not have a significant personal history of neurological or psychiatric disorders. This study was approved by the institutional review board of Carnegie Mellon University, and informed consent was obtained from all participants.
Stimuli for EEG Studies
In Experiment 1, stimuli were 160 grayscale pictures of everyday objects downloaded from the Bank of Standardized Stimuli (Brodeur, Guérard, & Bouras, 2014; Brodeur, Dionne-Dostie, Montreuil, & Lepage, 2010). Each image was divided into 4, 16, 64, or 256 squares that were randomly rearranged, resulting in five levels of scrambling (intact to most scrambled: S0, S4, S16, S64, S256), for a total of 800 distinct stimuli. Representative stimuli are shown in Figure 1A. Each version of each stimulus was viewed twice during the EEG procedure (1,600 images total). Each object occupied a centrally presented square subtending 6.5° of visual angle and was presented on a gray background. A central fixation cross (0.5°) was always present.
In Experiment 2, we constructed stimuli that selectively stimulated the parvocellular visual pathway. First, we created binary line drawings from all 160 stimuli used in Experiment 1 using the “Find Edges” filter in Adobe Photoshop. Then, for each participant, each image was converted to a binary isoluminant red/green image using red/green values calibrated via heterochromatic flicker photometry, previously described by Almeida et al. (2013). This process was done on a participant-by-participant basis. This approach has been successfully used to drive activity in the parvocellular visual pathway (Almeida et al., 2013; Kveraga et al., 2007); hence, we refer to this manipulation as “P-biased.” Each image was then randomly box scrambled using 2, 4, 9, and 64 pieces, resulting in five levels of scrambling (intact to most scrambled: S0, S2, S4, S9, S64). We used a smaller number of boxes for the scrambling procedure in this experiment than in Experiment 1 because the objects were more difficult to recognize than those used in Experiment 1. We sought to approximate equivalent psychophysics curves across all experiments (see Validation of stimuli below and Figure 1B) so that differences in EEG signal were not simply a result of substantially different recognizability across stimulus types. As in Experiment 1, each version of each stimulus was presented twice using the same visual angles. Importantly, we use line drawings in Experiments 2 and 3 because output of stimulus calibration for both of the latter experiments is binary in nature.
In Experiment 3, we constructed stimuli that selectively stimulated the koniocellular visual pathway. We used the line drawings created for Experiment 2 with the exception of a small subset of 20 images that were particularly challenging to recognize, even when completely intact, when calibrated for the koniocellular pathway. For this small subset, we replaced these images with an additional 20 line drawings created from images in the Bank of Standardized Stimuli image set (Brodeur et al., 2014) and confirmed that participants' object naming accuracy was highly similar to that in the previous two experiments (see Figure 1B). To create tritanopic stimuli that selectively activate the koniocellular pathway, we used the blue values calibrated on a participant-by-participant basis using the Cavanagh method (Cavanagh, Adelson, & Heard, 1992), previously described by Almeida et al. (2013). The stimuli were calibrated individually for each participant. The blue line drawings were placed on a bright yellow background, effectively saturating the responses of the red and green cones and the responses of the rods. The Cavanagh method has been implemented to drive activity selectively in the koniocellular visual pathway; hence, we refer to this condition as the “K-biased” condition. As in Experiment 2, all 160 stimuli were randomly box scrambled using 2, 4, 9, and 64 pieces, resulting in five levels of scrambling (intact to most scrambled: S0, S2, S4, S9, and S64). Again, each version of each stimulus was presented twice in the EEG experiment using the same visual angles.
Validation of Stimuli
To confirm that our experimental manipulations induced decrements in recognition as the scrambling level of the stimuli increased and that this held roughly to an equivalent degree across the three stimulus types, we conducted three behavioral experiments. A single stimulus was presented in the center of a computer screen for 400 msec (same duration as EEG procedure), and participants were required to identify and name each display. Participants completed five blocks of 160 trials. For all experiments, the level of scrambling was blocked so that each block contained only one version (scramble level) of each object and the total number of images at each level of scrambling was balanced across blocks. Participants were correct if they gave any appropriate name for a given object, including subordinate or basic level object names (i.e., “ball” or “volleyball” for picture of a “volleyball”). The results of the naming task for all three experiments are summarized in Figure 1B.
For the grayscale stimuli, 15 of the 20 participants who completed the first EEG experiment named the objects in 800 pseudorandomly presented images (160 objects × 5 levels of scrambling) across five blocks. Participants completed this object naming task several months after the EEG task. As in our analysis of EEG data, we obtained a regression coefficient for each participant by regressing object naming accuracy onto the level of scrambling. We then used a t test to evaluate whether the regression coefficients for all participants differed significantly from zero, and we found that they were highly significant (95% CI [−4.29, −3.94]). These results clearly show an inverse relationship between object naming accuracy and scrambling level. That is, naming accuracy decreases as the level of scrambling increases.
Eighteen participants naive to the EEG procedure completed the same stimulus validation procedure, but now requiring recognition of the P-biased stimuli. The same analysis used for the grayscale stimuli revealed that, for the P-biased stimuli, participant regression coefficients also differed significantly from zero, 95% CI [−4.92, −4.49]. Last, 14 naive participants completed the stimulus validation procedure with the K-biased stimuli. Again, the analysis revealed that the group of regression coefficients differed significantly from zero (95% CI [−4.50, −4.17]). We aimed to recruit about the same number of naive participants, for each of the latter two validation procedures, as we were able to recruit back from Experiment 1.
In a more direct comparison between experiments' difficulty, a repeated-measures ANOVA using Scrambling level (S0, S1, S2, S3, S4) as a within-participant factor and Experiment (grayscale, P-biased, K-biased) as a between-participant factor revealed no main effect of Experiment, F(2, 44) = 3.29, p = .05, but an interaction between Level and Experiment, F(4, 176) = 1832.7, p < .001. Post hoc evaluation of this interaction reveals no systematic pattern; for example, naming accuracy was higher at Level 2 of scrambling for grayscale than for either P-biased, t(31) = 2.86, p = .01, or K-biased, t(27) = 3.31, p < .01, stimulus type, but at Level 3 (grayscale vs. P-biased: t(31) = 0.62, p = .54) and Level 4 (grayscale vs. K-biased: t(27) = 0.23, p = .82), this pattern was not replicated. This lack of a clear widespread difference of one stimulus type suggests that object perception was not detrimentally affected by any one stimulus manipulation but, rather, that accuracy as a whole was relatively similar across the experiments. In summary, this analysis revealed largely similar psychophysics curves for P- and K-biased stimuli as with grayscale stimuli; in all cases, perceptual performance decreased significantly as the level of scrambling increased.
Images were presented with MATLAB PsychToolBox (Kleiner et al., 2007; Brainard, 1997) on an 18-in. CRT monitor with a 100-Hz refresh rate. On each trial, an image appeared for 400 msec with a 600- to 1000-msec variable ISI during which participants maintained fixation. Concurrent EEG was collected from 128 channels with a BioSemi 10–20 system (Cortec Solutions) at a sampling frequency of 512 Hz, using an active feedback circuit with the standard BioSemi Common Mode Sense and Driven Right Leg electrodes and no reference electrode. Simultaneously, activity was recorded from two vertical and two horizontal eye electrodes.
Participants completed an orthogonal fixation color change task in all three experiments. They were instructed to respond with a button press if the central fixation cross changed, during stimulus onset, from black to green in Experiment 1 or from black to blue in Experiments 2 and 3. A random 10% of trials contained fixation color changes. These trials were excluded from the analysis, as were any other trials containing a button press. As in the stimulus validation procedure, the level of scrambling was blocked so that each block contained only one version (scramble level) of each object and the total number of images at each level of scrambling was balanced across blocks.
Data were preprocessed in EEGLAB (Makeig, Debener, Onton, & Delorme, 2004) and ERPLAB (Lopez-Calderon & Luck, 2014). EEG data were rereferenced to the average of all 128 scalp electrodes and bandpass filtered from 0.1 to 40 Hz. Then, channels with excessive noise observed during EEG recording were removed and interpolated using a spherical head model. Eye blinks were removed by correlating individual independent component analysis components with horizontal and vertical eye channels. The EEG data were then divided into epochs (−100 to 500 msec poststimulus) and baseline corrected (−100 to 0 msec), and individual epochs were rejected by a sliding window peak-to-peak artifact detection algorithm if they contained artifacts within −100 to 500 msec poststimulus time window. Remaining epochs were averaged for each condition, yielding about 290 epochs per condition per participant.
EEG Amplitude Analysis
The analysis of the EEG data was done in sensor space, encompassing all 128 scalp electrodes. Given the differences in the visual stimuli, we completed the analysis for each EEG experiment separately at first and then compared the findings across the experiments. To test our hypotheses, we conducted four analyses in total, two within-experiment analyses and two between-experiment analyses.
Within each data set, in the within-experiment peak ERP analysis, we extracted peak amplitudes for each participant at four commonly studied ERP peaks (C1, P1, N1, and P2) using the ERPLAB toolbox. For each peak, we extracted the absolute peak amplitude (maximum or minimum) corresponding with peak polarity (i.e., maximum positive peak for P1). For each waveform, we utilized a time window commonly found in the literature (Luck, Woodman, & Vogel, 2000; Vogel & Luck, 2000; Mangun, 1995; Clark, Fan, & Hillyard, 1994; Luck & Hillyard, 1994) to capture the peaks of the ERP waveform averaged across participants in the grayscale condition (see Figure 2A): C1 (60–100 msec), P1 (100–150 msec), N1 (140–200 msec), and P2 (180–260 msec). The same time windows were used in the other two experiments. At each peak, we generated regression coefficients for each participant at each electrode by regressing the peak amplitude onto the level of scrambling. We then conducted significance testing for shape sensitivity by comparing the regression coefficients across all participants to the null hypothesis (regression coefficient = 0) using a cluster-corrected permutation test (1,000 permutations, α level = .05), separately for each ERP peak. For this analysis and all permutation tests described below, we used the ft_timelockstatistics command in the FieldTrip toolbox (Oostenveld, Fries, Maris, & Schoffelen, 2011). This approach assumes that significance should occur in spatially contiguous areas, such that only when sufficient number of neighboring electrodes exhibit significance does the cluster of electrodes then become significant, and thereby protecting against inflated Type I error. Clusters were defined as each electrode and its nearest 25 neighbors (results were identical with 20 and 30 neighbors). The results are summarized in Figures 2C–4C. This analysis was conducted separately for each EEG experiment. Our analysis is spatially unbiased and therefore represents a methodological advance from traditional peak ERP analyses that focus only on a few electrodes, oftentimes defined post hoc.
Because we could not predict, a priori, whether stimulus manipulations for Experiments 2 and 3 would evoke typical ERP peaks, or any peaks at all, we also carried out a within-experiment sliding window approach in which we examined shape sensitivity over the entire poststimulus epoch to document the full temporal evolution of shape processing without bias to specific ERP time points. As with the peak analysis above, this was done separately for each stimulus condition. We extracted mean amplitude of the ERP waveforms within a 20-msec sliding window (10-msec steps) from the onset of the stimulus to 500 msec poststimulus. Then, we calculated regression coefficients for each participant at each electrode and then performed the same cluster-corrected permutation test as above (1,000 permutations, α = .05) to compare data across participants to chance (regression coefficient = 0) at each step of the sliding window. Results for each stimulus condition are displayed in Figures 2B–4B.
To test our hypothesis that shape sensitivity emerges earlier in the dorsal pathway, relative to the ventral pathway, we carried out analyses directly comparing results from the P- and K-biased experiments. We conducted a between-experiment, independent-sample, cluster-corrected permutation test (permuting the experiment labels, 1,000 permutations, α = .05) on all electrodes at each time point for the P- and K-biased stimulus manipulations. The interpretation that follows is that any time point at which a significant difference is found reflects differential shape computations between Experiments 2 and 3. In other words, a significant finding indicates that biasing processing to either the dorsal or ventral pathway alone results in shape processing that is distinct from that arising in the other pathway, at a given time point.
To confirm and extend findings from Analysis C and to test our a priori hypotheses about the earliest emergence of shape sensitivity, we conducted a split-half correlation analysis at each of the earliest time points at which significant shape processing was found in Experiments 2 and 3 (50–100 msec). Here, we tested whether the correlation between the EEG responses of participants within a single experiment is greater than between the two experiments. First, we randomly divided participants from P- and K-biased experiments into two groups each. We then created average waveforms for each half group of participants. Thereafter, we correlated mean amplitudes, from a given sliding window, at all 128 electrodes of one average waveform with those from another average waveform from half groups of participants within an experiment. We then obtained a correlation value between experiments by sampling a random half of participants from each experiment (2 and 3) and calculating the correlation between the peak amplitude (at 128 electrodes) from the average waveforms of the P-biased condition with those from the K-biased condition. This process was iterated 1,000 times for random half groups of participants within and between experiments. This process yielded three correlation values: P-biased within experiment, K-biased within experiment, and between P- and K-biased experiments. We tested for significance by comparing the 95% confidence intervals for between-experiment correlations with the mean correlations within experiments. By comparing the correlations within a single experiment with correlations between the two experiments, we can establish the degree to which variability in shape sensitivity is distinct between the two experiments. That is, if the correlation within experiments is greater than the correlation between experiments, there is distinct shape-sensitive processing between experiments at that given time point.
The results from Experiments 1–3 are summarized in Figures 2–4. First, we describe the results from each experiment separately and then compare the EEG profiles elicited by the critical P- and K-biased manipulations. The dependent measure, the index of shape sensitivity calculated by regressing the amplitude of ERP responses onto the level of scrambling for each condition at the various time points, is the same dependent measure used successfully to measure shape sensitivity previously (Freud, Culham, et al., 2017). Additional data from the grand-averaged waveform for each experiment have been visualized across a wider array of electrodes in Figures 6–8.
Experiment 1 revealed a broad pattern of significant shape-sensitive activity across time and across a wide distribution of scalp electrodes (Figure 2A and C). First, in the peak ERP analysis, the cluster-corrected permutation test yielded significant clusters (α = .05) of shape sensitivity across all four peaks tested. The C1 peak, localized to the most posterior central electrodes, likely emerges from primary visual cortex (Clark et al., 1994). This early sensitivity might not reflect shape sensitivity per se but rather sensitivity to edges or high spatial frequency (i.e., greater activation for the more scrambled images; see Freud, Culham, et al., 2017; Lerner et al., 2001), in contrast to findings at later time windows. Note that, in contrast to the fMRI signal, EEG amplitude can be either positive or negative and therefore EEG shape sensitivity can also be positive or negative. For example, greater slope with a more positive signal amplitude for intact objects and then linearly decreasing with scrambling level results in positive shape sensitivity. Greater slope with a more negative signal amplitude for intact objects and then linearly increasing with scrambling level results in a negative shape sensitivity. Shape sensitivity was widely distributed across the scalp at the P1 peak and became decreasingly posterior and right lateralized in subsequent N1 and P2 peaks, respectively. The sliding window analysis corroborated the ERP peak analysis as shape sensitivity was significant as early as the 50-msec time window after the onset of the stimulus, continued until 160 msec, and then resumed significance at 240 msec poststimulus for the remainder of the time window examined. Additional electrode plots are shown in Figure 6 at the end of this article. Although we analyzed the data from the grayscale stimuli as a benchmark establishing the feasibility of the current paradigm in the setting of EEG, the key comparison is between the P- and K-biased stimuli.
In Experiment 2, the isoluminant red-green stimuli appeared to elicit restricted clusters of shape sensitivity (Figure 3), relative to Experiment 1. The cluster-corrected permutation test yielded no significant C1 shape sensitivity, despite the presence of C1 waveform morphology. In contrast, the P1 peak did evince shape sensitivity in the most posterior central electrodes. There were no significant clusters in the N1 peak. Finally, the P2 peak had several significant negative clusters located somewhat more anteriorly. The sliding time window analysis revealed significant clusters from 70 to 90 msec, then from 120 to 150 msec, and, finally, from 200 to 230 msec. The results from ERP peak and sliding window analysis do overlap, although not completely, suggesting that the sliding window analysis may, in fact, be a better measure of shape sensitivity, with clearer demarcation of individual differences in peak latency. Additional electrode plots are shown in Figure 7 at the end of this article.
Finally, in Experiment 3, significant clusters of shape sensitivity were evident in both the ERP peak and sliding window analyses (Figure 4). Peak analyses revealed no significant clusters at the C1 or P2 peak, which may reflect an ERP morphology highly divergent from that of the other two experiments. Indeed, the primary koniocellular input to cortex is not delivered entirely through V1, and so the absence of a C1 response is not surprising. However, there were significant clusters of shape sensitivity at both P1 and N1 peaks in centrally located electrodes. The sliding window analysis revealed significant activity as early as 50–60 msec, continuing from 110 to 250 msec, and finally from 360 to 480 msec. The peak and sliding window analyses corroborate each other, and again, the sliding window analysis appears more sensitive to small windows of shape sensitivity. Additional electrode plots are shown in Figure 8 at the end of this article.
The stimulation of each of the two separate thalamocortical visual inputs revealed distinct patterns of shape sensitivity within the first 500 msec of processing. Our EEG analyses in Experiment 1 revealed a robust spatio-temporal pattern of shape processing, beginning as early as 50 msec and persisting across nearly the entire epoch examined. In contrast, and although the overall ERP waveform morphology was similar to that of the first experiment, with respect to the presence of typical ERP peaks, visual information projected through the parvocellular pathway evoked substantially more selective windows of shape sensitivity. Importantly, the early emergence of shape sensitivity observed in Experiments 2 (at 50 msec) and 3 (at 70 msec) does not likely reflect the activity of primary visual cortex, as is likely the case in Experiment 1. Figure 5 shows the first time window with significant shape sensitivity from each experiment. Note that the first window with reliable shape sensitivity from Experiment 1 (50 msec) includes the most posterior centrally located electrodes. In contrast, the earliest time windows containing significant shape sensitivity in Experiments 2 (70 msec) and 3 (50 msec) are located less posteriorly and do not contain any of the electrodes that typically pick up primary visual cortex activity. Finally, stimulation of the koniocellular pathway also elicited more selective but significant shape sensitivity, but perhaps, of greatest relevance is the apparently different morphological pattern of the ERP waveform in the P-biased versus K-biased studies.
In addition to the above sliding window permutation tests conducted within each of the experiments, we also conducted a more conservative permutation test correcting over both sliding windows (time) and electrodes (space) for each experiment, separately. The analysis of Experiment 1 revealed two significant clusters from 40 to 190 and 240 to 500 msec poststimulus. The same analysis in both Experiments 2 and 3 failed to reveal any clusters robust enough to survive this highly conservative correction. We did not continue with corrections over both time (sliding window) and space (electrodes) in between-experiment analyses because we found this correction to be overly conservative. That is, in contrast to both traditional ERP peak analyses and our sliding window analyses that correct across electrodes, this additional correction results in insensitivity to more rapidly changing dynamics of the EEG signal. Said another way, perturbations may occur within one or two sliding windows, such as an ERP peak, but be subsequently corrected over. Because this approach fails to pick up meaningful electrophysiological differences observed within specific ERP peaks, our inferences of within-experiment results reflect analyses without this additional correction.
Thus far, we have described the temporal and topographic profile associated with the three different inputs separately, although we have drawn some qualitative comparisons between them. A more exact understanding of the relative contribution of dorsal and ventral pathways to shape processing requires a direct comparison between the P- and K-biased experiments. To determine whether there were differences in the temporal dynamics of shape sensitivity in the P- versus K-biased experiments, we conducted an additional independent-sample, cluster-corrected permutation test on shape sensitivity at each sliding window. The permutation was conducted using the data from both Experiments 2 and 3 and randomly shuffling the labels of the experiments so as to create a null distribution (see Methods for details). The results revealed that shape sensitivity was significantly different (i.e., those time points at which probability of a cluster given the permuted distribution is below the alpha level of .05) between P- and K-biased experiments at several time points: from 60 to 200 msec, from 220 to 240 msec, and, finally, from 380 to 410 msec, suggesting distinct patterns of shape processing during these windows.
Although the results from the above analysis show differences in shape sensitivity in the two pathways, the origins of the differences remain somewhat ambiguous. In addition, as noted previously, at later time points, there is invariably widespread propagation of signals between visual pathways. For this reason, we focused our final analysis on the earliest signals to determine whether the shape sensitivity found in the K-biased condition simply reflects information that differs from that activated in the P-biased condition. To this end, we completed an additional analysis in the first few sliding windows where shape sensitivity emerged, and we examined whether the information contained in the ERP signals between the two experiments were less similar to each other than the information contained within each of the two experiments. If the information between the experiments is overlapping, no differences should be observed in the between- and within-experiment correlations. Consistent with the first between-experiment analysis above, the split-half correlation analysis revealed correlations that were significantly higher within P-biased and within K-biased experiments than between them, particularly at time points in which significant shape sensitivity was found within Experiments 2 or 3 (see Table 1). These results demonstrate that P- and K-biased stimuli evoked unique shape processing in these early time windows and reinforce the claim that the patterns of significant shape sensitivity differ between dorsal and/or ventral pathways. Together with results from the between-experiment permutation test, these findings indicate that K-biased stimuli evoke shape sensitivity significantly earlier than P-biased stimuli and that the information during the early time points at which shape sensitivity emerges is not the same in the two visual pathways. Furthermore, and importantly, the early shape sensitivity of the dorsal pathway confirms that the dorsal computations are not a product of the representations derived by the ventral pathway.
Higher correlations suggest greater consistency among participants and, therefore, more similar underlying processes. The final column shows 95% confidence intervals around the mean correlation in between-experiment comparisons. Gray-shaded boxes are those with significant cluster permutation tests done separately, within Experiments 2 and 3. Asterisks in the table signify that within-experiment correlations are outside null distribution of between-experiment correlations.
p < .05.
In summary, Experiment 1 reveals broad spatio-temporal patterns of shape sensitivity when both dorsal and ventral pathways receive input from the retina. Experiments 2 and 3 also reveal reliable shape processing, but this occurs when the visual input is propagated to ventral and dorsal visual pathways largely independently, and the spatio-temporal signatures differ across these two experiments. In addition, shape-sensitive information appears to emerge earlier when input is propagated predominantly (perhaps solely, initially) to the dorsal pathway compared with when input is propagated predominantly to the ventral pathway. Finally, the shape sensitivity that emerges earliest in each of the two pathways appears to reflect independent or nonredundant processing.
Previous research has demonstrated that both dorsal and ventral visual pathways represent shape information, supporting the hypothesis that the two anatomically distinct cortical pathways exhibit a graded—rather than absolute—specialization and division of labor (Freud, Culham, et al., 2017; Freud et al., 2016). The primary aim of the current study was to evaluate the origins of the shape sensitivity signals in dorsal cortex given that, with the dense connectivity between the pathways, these signals might simply reflect the cascade of information propagated from ventral cortex. We therefore examined whether the signals in the two pathways shared the same spatio-temporal profile or not, with a specific focus on whether shape selectivity was evident in one pathway earlier than in the other, and whether the information in the two pathways was identical or not. An overlap in temporal dynamics and in information represented would favor an interpretation of dorsal cortex being the recipient of the shape selectivity from the preeminent ventral pathway. In contrast, differences in profile, temporal dynamics, and information would implicate a dissociation between the pathways, specifically with dorsal signals computed independent of ventral signals.
In addition to establishing the feasibility of the approach and documenting the spatio-temporal cortical signature in response to grayscale stimuli, in two additional experiments, we manipulated the stimuli to favor differential processing in the dorsal pathway (koniocellular) or the ventral pathway (parvocellular). Activation of dorsal and ventral pathways with grayscale stimuli as inputs revealed the most robust spatial and temporal indices of shape sensitivity of the three inputs tested. Biasing activity in the parvocellular pathway, which also synapses in primary visual cortex but then propagates almost exclusively through the ventral visual pathway, elicited the most limited pattern of shape sensitivity of the three pathways tested. This was unexpected given substantial evidence demonstrating the ventral visual pathway's involvement in object recognition (e.g., Konen, Behrmann, Nishimura, & Kastner, 2011; Goodale, Milner, Jakobson, & Carey, 1991), an issue we raise for further discussion below. Biasing activity in the koniocellular pathway, which synapses primarily in V5/MT of the dorsal pathway, also elicited a broad temporal pattern of shape sensitivity, with the earliest emergence of activity similar to that of the grayscale rather than to the P-biased condition. The earlier activation of dorsal cortex and the reduction in the signal correlation in the between- versus within-experiment comparisons suggest that the early signals processed in dorsal and ventral cortices favor the interpretation that the shape representations are independent. Of note, that the signal in dorsal cortex emerges earlier than that in ventral cortex challenges the notion that the activation in dorsal cortex is simply a consequence of ventral signals being projected to (or “uploaded to”; Xu, 2018) dorsal cortex. Instead, these findings indicate that shape selectivity is computed at least partially independently in the two visual pathways.
As noted in the Introduction, the apparent segregation of the signals in the two visual pathways is compatible with the finding that individuals with damage to ventral cortex are nevertheless sensitive to properties or shapes (e.g., whether a 3-D object is legitimate or possible vs. impossible; Freud, Ganel, et al., 2017). This retained sensitivity was even observed in a patient with very extensive bilateral occipito-temporal lesions whose perceptual performance resembles that of the well-known patient, D. F. (Marotta, Behrmann, & Goodale, 1997). The results are also consistent with findings from an imaging study, which used the same grayscale objects as those adopted here and which, using representational similarity analysis, showed that some dorsal regions had similar representational bases to some ventral regions (Freud, Culham, et al., 2017).
The Parvocellular Pathway Response
Before considering the implications further, there are some aspects of the data that warrant further clarification. Perhaps surprisingly, given the key role of ventral cortex in object perception (for a recent review, see Weiner, Natu, & Grill-Spector, 2018), when the inputs biased processing to the ventral pathways (P-biased), shape selectivity was more restricted temporally than was true for either the grayscale or koniocellular cases. There may be multiple explanations for this seemingly unexpected result. First, close examination of the findings shows that the temporal pattern of activity for the intact object condition differs slightly from the pattern one might predict were there a linear gradient from the remaining four scrambled conditions (see Figure 2A). For example, in the P100 peak, the intact object does not have the largest amplitude, as one might expect, despite a clear decrease in amplitude for each additional level of scrambling. This discontinuity suggests that intact objects may be processed distinctly from scrambled objects in the ventral pathway. Consistent with this, recent evidence has revealed that nonlinearities for whole-object perception exist along the ventral pathway (Landi & Friewald, 2017). Furthermore, within the ventral pathway, certain regions exhibit preferential activity to intact objects, and this activity is insensitive to low-level image manipulation (Malach et al., 1995). Although beyond the scope of the current study, future efforts should explore the extent to which dorsal and ventral pathways preferentially treat intact over scrambled objects. Second, although dorsal and ventral pathways may derive independent representations in the earliest stages of processing, input to dorsal cortex itself (and to ventral cortex) may be critical in eliciting a “typical” shape-sensitive response from the ventral pathway during object perception, given reciprocal connections between pathways. The latter hypothesis is supported by findings from our K-biased stimulus condition (Experiment 3). We also note that these hypotheses are not mutually exclusive.
The Koniocellular Pathway Response
The interpretation of our K-biased results should also be considered in the context of other established findings. Given that the koniocellular pathway synapses both in Layer 1 of primary visual cortex and directly in the dorsal pathway (V5/MT), the signal in dorsal cortex can arise from either origin (Dobkins, 2000; Hendry & Reid, 2000; Morand et al., 2000; Casagrande, 1994). To our knowledge, the current study is the first EEG study conducted that uses K-biased stimuli to explore object perception, and so no benchmark exists against which to compare our results. However, a closer look at the morphology of the ERP waveform may be revealing. First, there is no discernable C1 peak in the central posterior electrode shown in Figure 4A, which suggests that most of the response measured from koniocellular input is not entirely through primary visual cortex, in contrast with the other two experiments (compare Figure 4A with Figures 2A and 3A). Second, the overall morphology of the waveform is substantially different from that elicited by the grayscale and P-biased stimulus conditions. If most koniocellular input traversed primary visual cortex, one would expect to see ERP morphology that reflects a roughly similar flow of information to that observed in the other experiments. Additional evidence from one of the few human EEG studies done exploring koniocellular activity in the cortex does demonstrate that K-biased stimulation results in rapid activation of the dorsal visual pathway (Morand et al., 2000). Our findings replicate this result. In addition, some neuropsychological data support the results showing that object perception is still possible despite the absence of primary visual cortex (Mundinano et al., 2017). The residual shape sensitivity possibly implicates the koniocellular projections to MT and dorsal cortex more generally. Together, this evidence suggests that the activity elicited by the K-biased stimuli largely reflects activity generated from thalamocortical synapses in V5/MT of the dorsal visual pathway.
An alternative explanation of shape sensitivity in the dorsal pathway that cannot be definitively ruled out is a general coherence function. Motion coherence specifically drives activity in V5/MT, and object coherence can selectively activate activity in other dorsal regions (Braddick, O'Brien, Wattam-Bell, Atkinson, & Turner, 2000), suggesting that the dorsal stream may include some general coherence function that could yield the pattern of results observed in Experiment 3. Generally, the slope of responses could be because of a general coherence mechanism decreasingly responding to increasingly scrambled objects. However, the same objects used here evoked highly similar shape sensitivities in dorsal and ventral regions using fMRI (Freud, Culham, et al., 2017), suggesting that a general dorsal coherence mechanism cannot account for all findings but may play a role in the processing trend between intact objects and increasingly scrambled objects. That is, the similarity between findings in the dorsal and ventral pathways as measured with fMRI suggests that, if a general coherence function where the mechanism driving dorsal responses, a similar coherence mechanism would need to be driving ventral responses in the same way.
Representations in Dorsal Pathway
We have argued for neural representations of shape information in dorsal cortex. The exact functional role of the dorsal pathway in shape perception or in object recognition, more generally, is not well understood. Recent studies in humans (Freud, Culham, et al., 2017; Bracci & Op de Beeck, 2016; Konen & Kastner, 2008) and in nonhuman primates (Janssen et al., 2000, 2008; Durand et al., 2007) have revealed sensitivity of the dorsal pathway to shape information. Moreover, neuropsychological (Freud, Ganel, et al., 2017) and electrophysiological (Van Dromme et al., 2016) studies have demonstrated that these dorsal representations are, at least partially, independent from ventral representations (but see Xu, 2018, for a different view).
These independent neural representations in the dorsal pathway may facilitate visual object recognition via top–down processes, perhaps from the OFC, where object recognition elicits differential activity within 50 msec, earlier than that in the ventral temporal cortex (Bar et al., 2006). A partially processed version of the input may be rapidly projected to PFC to constrain the interpretation of the image, before information processing in the ventral cortex (Bar, 2003). One candidate pathway through which this information is propagated to PFC is the dorsal visual pathway. The results from the current study are consistent with these findings and provide novel evidence for the differential time courses of shape processing along the two pathways. Importantly, the rapid emergence of shape sensitivity in Experiment 3 in which the koniocellular system was targeted (Morand et al., 2000; see also Caprara, Premereur, Romero, Faria, & Janssen, 2018) supports the idea that the dorsal representations are not epiphenomenal of the representations derived by the ventral pathway.
In conclusion, the current study offers a characterization of the temporal dynamics of shape processing in the dorsal and ventral visual cortices. The evidence provided here supports the theory that the dorsal and ventral pathways exhibit graded, but not absolute, division of labor in the human visual system as far as object perception is concerned. Last, the earlier emergence of shape sensitivity in the dorsal pathway and the nonoverlapping signals between the dorsal and ventral EEG signals suggest that the shape selectivity in dorsal cortex is not simply a reflection of the signals from the ventral pathway. Together, these findings suggest (at least partial) independence of shape representations between pathways.
This work was supported by the Richard King Mellon Foundation Presidential Fellowship to E. C. We thank Brad Mahon for stimulus calibration scripts and Yingli Sieh and Jieming Li for assistance in EEG data collection. We also thank Adrian Nestor for helpful comments on data analysis.
Reprint requests should be sent to Elliot Collins, Carnegie Mellon University, 5000 Forbes Ave., Pittsburgh, PA 15213, or via e-mail: email@example.com.