The visual system has been commonly subdivided into two segregated visual processing streams: The dorsal pathway processes mainly spatial information, and the ventral pathway specializes in object perception. Recent findings, however, indicate that different forms of interaction (cross-talk) exist between the dorsal and the ventral stream. Here, we used TMS and concurrent EEG recordings to explore these interactions between the dorsal and ventral stream during figure-ground segregation. In two separate experiments, we used repetitive TMS and single-pulse TMS to disrupt processing in the dorsal (V5/HMT+) and the ventral (lateral occipital area) stream during a motion-defined figure discrimination task. We presented stimuli that made it possible to differentiate between relatively low-level (figure boundary detection) from higher-level (surface segregation) processing steps during figure-ground segregation. Results show that disruption of V5/HMT+ impaired performance related to surface segregation; this effect was mainly found when V5/HMT+ was perturbed in an early time window (100 msec) after stimulus presentation. Surprisingly, disruption of the lateral occipital area resulted in increased performance scores and enhanced neural correlates of surface segregation. This facilitatory effect was also mainly found in an early time window (100 msec) after stimulus presentation. These results suggest a “push–pull” interaction in which dorsal and ventral extrastriate areas are being recruited or inhibited depending on stimulus category and task demands.
The visual system has been divided into a manifold of separate cortical regions (Felleman & Van Essen, 1991). In the progress of understanding this vast collection of distinct specialized cortical areas, the visual system has been broadly divided into two functionally and anatomically separate processing streams. The dorsal stream mainly processes visuospatial information, such as motion, distance, and location (the “where” or “vision for action” pathway), whereas the ventrally spreading “what” or “vision for perception” pathway specializes in object recognition (Goodale & Milner, 1992; Mishkin, Ungerleider, & Macko, 1983). Recently, however, it has been debated to what extent the dorsal and ventral streams are functionally segregated (Schenk, 2012; de Haan & Cowey, 2011; Cardoso-Leite & Gorea, 2010; Schenk & McIntosh, 2010; Doniger, Foxe, Murray, Higgins, & Javitt, 2002).
In the past decade, a growing number of studies show that traditional “what” and “where” information is able to modulate activity in both ventral and dorsal cortical areas (Hesselmann & Malach, 2011; Konen & Kastner, 2008; Doniger et al., 2002; Braddick, O'Brien, Wattam-Bell, Atkinson, & Turner, 2000). In addition, competitive mechanisms between dorsal and ventral systems have been revealed by varying task demands (Majerus et al., 2011; Jokisch & Jensen, 2007) or by applying TMS over motion-sensitive area V5/HMT+ (Walsh, Ellison, Battelli, & Cowey, 1998). In a series of experiments, Ellison and Cowey demonstrated cooperation and differentiation of the dorsal and ventral stream by disrupting activity in dorsal (posterior parietal cortex) and ventral (lateral occipital area [LO]) regions during a visuospatial discrimination task (Ellison & Cowey, 2006, 2007, 2009). These findings reveal collaborative or competitive interactions between dorsal and ventral extrastriate regions, suggesting a less absolute segregation between dorsal and ventral pathways.
Here we used repetitive TMS (rTMS) to explore how dorsal and ventral cortical areas contribute to figure-ground segregation. To investigate the contribution of dorsal and ventral regions during figure-ground segregation, we manipulated neural activity in either V5/HMT+ or object-selective region LO while concurrently recording EEG signals during a motion-defined figure discrimination task. The design of our stimuli and task set-up allowed us to disentangle relatively low-level (figure boundary detection) from higher-level (surface segregation) processing steps during figure-ground segregation (Wokke, Sligte, Scholte, & Lamme, 2012; Scholte, Jolij, Fahrenfort, & Lamme, 2008; Vandenbroucke, Scholte, Van Engeland, Lamme, & Kemner, 2008; Heinen, Jolij, & Lamme, 2005).
In a second experiment, we focused on the temporal dynamics of possible dorsal/ventral stream interactions using single-pulse TMS. This time, we briefly disrupted area V5/HMT+ and LO at several time points after stimulus presentation. Combined results of both experiments show that disruption of V5/HMT+ deteriorated performance associated with higher-level stages of motion-defined figure-ground segregation. Reduced performance scores were mainly found when V5/HMT+ was disrupted in an early (100 msec) time window after stimulus presentation. Surprisingly, disruption of LO resulted in improved performance scores and enhanced neural correlates of motion-defined figure-ground segregation. This facilitatory effect was again mostly found in an early time window (100 msec). These results suggest a “push–pull” mechanism in which dorsal and ventral extrastriate areas are being recruited or inhibited depending on the type of stimulus and current task demands.
EXPERIMENT 1 METHODS: rTMS-EEG
Seven undergraduate psychology students of the University of Amsterdam (seven women, aged between 19 and 24 years) participated in this study for financial compensation. All participants had normal or corrected-to-normal vision and were naive to the purpose of the experiments. Participants were screened before the experiments according to international guidelines (Rossi, Hallett, Rossini, & Pascual-Leone, 2009; Wassermann, 1998), and before the experiments participants gave their written informed consent. The ethics committee of the Psychology department of the University of Amsterdam approved all procedures. All participants were well trained in the experimental task and accustomed to TMS stimulation and EEG recordings. All participants already took part in a previous study (∼2,400 trials) performing the same task in combination with stimulation of the early visual cortex and EEG recordings.
We presented stimuli full screen (1024 × 768 pixels) on a 17-in. DELL TFT monitor with a refresh rate of 60 Hz. In the first experiment, the monitor was placed at a distance of ∼90 cm in front of the participant so that each centimeter subtended a visual angle of 0.64°. We instructed the participants to discriminate between a stack, frame, and homogenous stimulus (see Figure 1B and below for details; see Wokke and colleagues, 2012, for similar task design and EEG procedure). These three stimuli have been associated with different steps during figure-ground segregation (Wokke et al., 2012; Scholte et al., 2008; Vandenbroucke et al., 2008; Heinen et al., 2005), related to different levels of processing in early visual cortex (Wokke et al., 2012; Scholte et al., 2008; Vandenbroucke et al., 2008; Heinen et al., 2005; Zipser, Lamme, & Schiller, 1996; Lamme, 1995). Our stimuli in combination with rTMS and concurrent EEG recordings allowed us to study how different steps during figure-ground segregation and previously described neural correlates in early visual cortex (Pitts, Martínez, Brewer, & Hillyard, 2011; Scholte et al., 2008) were (differentially) affected by disruption of neural activity in V5/HMT+ and LO.
We used stimuli in which figure-ground segregation was achieved by relative motion of random dots (Wokke et al., 2012; Zipser et al., 1996). We created our stimuli by placing randomly distributed black and white dots (one pixel in size) across the screen. Each pixel had an equal probability of being black or white. A stimulus consisted of three regions: the background (17.99°; 24.8 cd/m2), the figure frame (3.23°; 24.8 cd/m2), and the inner figure (2.42°; 24.8 cd/m2). Stimulus presentation consisted of two screen refreshes (33.3 msec) in which the random dots were displaced one pixel per screen refresh in one of four directions (45°, 135°, 225°, or 315°). During the first screen refresh, the random dots were displaced one pixel in one of the four directions; during the second screen refresh, the dots were moved one pixel further in that same direction. Before and after stimulus presentation, the screen was filled with stationary random dots (Figure 1); stimulus presentation consisted merely of moving these dots
A homogenous stimulus was created by displacing the dots of all three stimulus regions coherently in one direction. We created the frame stimulus by displacing the dots of the frame region in a different direction than those of the background and inner figure (which were displaced in the same direction), so that a frame appeared to be floating over and moving in another direction than that of the background. The stack stimulus appeared when the dots of the inner figure region were displaced in one direction, the dots of the frame region in another direction and background dots in yet another direction, so that a “stack” of figures appeared to be moving against the background.
In all three stimuli, the pixels within each region did not cross their fixed border (Figure 1B). As a consequence, all stimuli produced the same amount of flicker because of (dis)appearing dots. Moreover, on average, all three stimuli contained the exact same strength and directions of motion of dots, so that motion energy was fully balanced between stimuli. Finally, stack and frame stimuli were perfectly balanced with respect to local motion contrast: both stimuli contained an equal amount of borders where motion was in orthogonal directions. The only difference between stack and frame stimuli is in the amount of figure surface that can be perceived: in the frame stimulus, only the frame region segregates from background; in the stack stimulus, both frame and inner figure region segregate.
Each trial started with a blank screen (1500 msec; 24.8 cd/m2) followed by a display filled with an equal amount of randomly distributed black and white dots with a red fixation dot placed in the center of the screen (0.15°; 2500 msec; see Figure 1A), after which the fixation dot turned green (0.15°; 250 msec). The stimulus was presented in the lower left corner of the fixation dot (off center: horizontal 7.7°; vertical 10.64°) for two screen refreshes (33.3 msec). After stimulus presentation, all dots remained in position and on screen. The trial ended when a response was given. Participants were instructed to discriminate between the three stimuli and press a left button on a button box placed at the left-hand side (left index finger) when they thought that a homogenous stimulus was presented, the left button on a button box placed on the right-hand side (right index finger) when they thought a frame was presented, and the right button on a button box placed on the right-hand side (right middle finger) when a stack was presented (response mappings counterbalanced between participants). Participants were instructed to keep their eyes fixated on the fixation dot while directing their attention toward the location where the stimuli were presented. Within each block, stimulus type was randomized and equally probable. Stimuli were presented using Presentation (Neurobehavioral Systems).
Because of rTMS limitations set by the local ethical committee and our limits set for the maximum amount of TMS coil movement because of head motion (see rTMS protocol below), data were gathered in an average of 18 rTMS-EEG sessions per participant (approximately 90 min per session) in which four experimental blocks per session were recorded, each containing 96 trials (resulting in ∼200 trials per condition per participant). After recording two blocks we cooled the coil for 15 min.
In the first ∼10 blocks of our experiment, we intermixed motion-defined stimuli with a texture-defined version of the same stimuli (for a detailed description of the texture-defined stack, frame, and homogenous stimuli, see Scholte et al., 2008). However, after several blocks, it became apparent that the texture-defined stimuli were not well suited for our experimental set-up (e.g., all participants had near perfect scores [mean = 98.3% correct, SD = 1.6] for stack stimuli, making it hard to find a behavioral TMS effect; Kammer & Nusseck, 1998). We continued the experiment excluding texture-defined stimuli and discarded all trials in which texture-defined stimuli were presented.
To disrupt processing in LO or V5/HMT+ (Figure 1C), we applied five pulses at 10 Hz over either right V5/HMT+ or right LO, 500 msec before stimulus presentation (Figure 1A). In this way, we created a short period after stimulus presentation in which neural activity in LO or V5/HMT+ was altered by rTMS (Romei, Gross, & Thut, 2010) and allowing us to record EEG signals without TMS pulses in the time window of interest (see below). We used a Magstim Rapid2 (Magstim Company, UK) stimulator and a 70-mm figure-of-eight coil. To determine the appropriate stimulation strength, we defined the phosphene threshold for each participant before starting the experiment. To define the phosphene thresholds, we increased the stimulator output while targeting areas V1/V2 until 50% of the (single) pulses resulted in the perception of a phosphene (eyes open in a dim lit room, fixating on a black screen). During the experiment, we used ∼80% of the phosphene threshold (mean = 56.4%, SD = 3.6% of stimulator output) to stimulate areas right LO and right V5/HMT+. To reduce fluctuation in phosphene threshold as much as possible, we asked participants before each session about their caffeine and alcohol intake in the last 24 hr and inquired about the quality of sleep they experienced the previous night.
To target right LO and right V5/HMT+ the coil was placed tangentially to the head using an fMRI-guided navigation system (ANT-Visor system). This navigation system makes use of functional and structural MRI data of each participant individually (for LO and V5/HMT+ mapping specifications, see below) enabling accurate positioning of the center of the coil over either right LO or right V5/MT. Accurately monitoring the position of the TMS coil with a navigation system has allowed researchers to even successfully target subregions of LO (LO1 and LO2) recently (Silson et al., 2013). During stimulation participants were seated in a chin rest for optimal stability while using a holder to fixate the coil (Rogue Research). We tracked the coil during each block, allowing a maximum coil displacement of 0.4 cm. If this limit was exceeded, all data from that block were discarded.
We added a control session in which we stimulated vertex to rule out any nonspecific behavioral effects of rTMS (i.e., because of noisy clicks or cutaneous stimulation). We used the same stimulator settings as during LO and V5/HMT+ stimulation and recorded 64 trials per condition (in four blocks). To keep the circumstances as similar as possible to the sessions in which we stimulated LO and V5/HMT+, an EEG cap was placed on the heads of the participants during vertex stimulation (although no actual EEG signals were recorded). Unfortunately, we were only able to obtain data from six of our seven participants during vertex stimulation, because of emigration of one of the participants.
In each block, we pseudorandomly intermixed rTMS trials with trials without stimulation, creating two rTMS conditions per target location (rTMS/no rTMS). Each block contained 50% trials in which we applied rTMS and 50% trials without rTMS.
LO and MT Mapping: fMRI
We targeted right V5/HMT+ or right LO using an fMRI-guided navigation system (ANT-Visor system). Therefore, we functionally mapped areas V5/HMT+ and LO using fMRI. To functionally map LO, we presented faces, houses, objects (bottles, chairs, and scissors) and phase-scrambled versions of the objects every 2 sec in blocks lasting 16 sec. Every block was presented four times. We made predictors by convolving the onset times of the stimuli from the different categories with a model of the HRF and fitting these to the data with the general linear model. To determine the location of LO, we contrasted houses, faces, and objects versus the scrambled versions of these objects (Grill-Spector & Malach, 2004). We further specified LO by subtracting overlapping regions of areas FFA (faces > houses and objects) and PPA (houses > faces and objects).
Area V5/HMT+ was mapped by comparing activity evoked by blocks of coherently moving dots with presentations of randomly appearing stationary dots. Each block lasted for a period of 16 sec (320 sec in total). During a 16-sec block of coherent movement, dots alternated motion direction (inward and outward) every 2 sec. Data were analyzed by means of a general linear model. We generated predictors by convolving the onset times of the moving stimuli and nonmoving stimuli with a model of the HRF. We fitted these predictors to the MRI data and generated a contrast between these two predictors (Dumoulin et al., 2000).
BOLD-MRI (GE-EPI, transversal slice orientation, repetition time = 2000 msec, echo time = 28 msec, field of view = 200 mm, matrix size of 112 × 112, slice thickness = 2.5, slice gap = .3, 28 slices, and a sense factor of 2.5) was recorded during presentation of stimuli (Philips, Achieva 3T). Stimuli were projected on a screen at the rear end of the scanner table and viewed via a mirror placed above the participant's head. The functional images were motion-corrected, slice time aligned, temporally smoothed with a Gaussian filter (FWHH of 2.8 sec), and high-pass filtered (0.01 Hz) in the temporal domain, without using spatial smoothing. The functional images were aligned to the structural image acquired at the end of each scanning session (T1 turbo field echo, 182 coronal slices, flip angle = 8, echo time = 4.6, repetition time = 9.7, slice thickness = 1.2, field of view = 256 × 256, matrix = 256 × 256).
On behavioral data, we performed repeated-measures ANOVAs on mean percentage correct, with factors rTMS Condition (LO, V5/HMT+, or no stimulation) and Stimulus Type (stack, frame, and homogeneous). Repeated-measures ANOVAs were also performed on mean RTs with factors rTMS Condition and Stimulus Type. RTs of less than 100 and greater than 1500 msec were excluded from all analyses.
EEG Measurements and Analyses
EEG was recorded and sampled at 1024 Hz using an ANT 64-channel system (ANT-ASA-Lab system of ASA). Sixty-four scalp electrodes were measured, as well as four electrodes for horizontal and vertical eye movements (each referenced to their counterpart). In Matlab (Mathworks, Natick, MA), we set EEG sample values to zero in an interval disrupted by the TMS pulses (−500 to −50 msec in relation to stimulus onset). Next we interpolated (using a spline interpolation) the EEG samples set to zero, so we were able to filter the data (Sadeh et al., 2011). We filtered the data using a high-pass filter of 0.5 Hz, a low-pass filter of 30 Hz, and a notch filter of 50 Hz, down-sampled to 256 Hz, and rereferenced to Cz. Non-rTMS-related artifacts such as eye movements were corrected for on the basis of independent component analysis (Vigário, 1997). All EEG data were visually inspected for artifacts (trials containing artifacts were removed from further analyses). Epochs between −50 to +750 msec around stimulus presentation were selected. To increase spatial specificity and to filter out deep sources, we converted the data to spline Laplacian signals (Perrin, Pernier, Bertrand, & Echallier, 1989). We baseline-corrected the data by subtracting the average sample value between −50 to 0 msec relative to onset of the stimulus from the data. Finally, all trials were averaged per condition. Because we recorded no-rTMS trials in both the LO and V5/HMT+ conditions, this resulted in twice as many trials in the no-rTMS condition compared with the LO and V5/HMT+ conditions. Therefore, we divided the no-rTMS data in odd and even trials. Next we collapsed the odd trials in which no rTMS was applied across the LO and V5/HMT+ conditions. For further analyses, we only used these odd trials in which no rTMS was administered, thereby balancing the amount of trials across all conditions. Otherwise differences between the different rTMS conditions could be because of an enhanced signal to noise ratio in the no-rTMS condition. All preprocessing steps were done using Brian Vision Analyzer (BrainProducts, Gilching, Germany), Matlab (Mathworks), and ASA (ANT-ASA-Lab, The Netherlands).
We created an a priori pooling of electrodes to increase the signal-to-noise ratio and decrease the amount of comparisons. We based our pooling (O2, POz, PO4, PO6, and PO8) on previous literature showing neural correlates of figure-ground segregation in these channels (Pitts et al., 2011; Scholte et al., 2008) and where we expected rTMS would have an effect (therefore focusing on right perioccipital channels).
To cancel out effects in our EEG data generated by the TMS pulses (i.e., not the TMS artifacts but the evoked neural activity from these pulses), we made use of a subtraction technique (Sadeh et al., 2011; Thut, Ives, Kampmann, Pastor, & Pascual-Leone, 2005) that also controls for the neural effect of local dot displacement: We subtracted ERPs on trials containing a homogenous stimulus from ERPs on trials containing a figure stimulus (stacks and frames collapsed; see Figure 3) for each rTMS condition separately (Scholte et al., 2008; Taylor, Nobre, & Rushworth, 2007). The resulting difference waves (figure-homogenous difference) now reflect activity related to processing of the figure without activity related to local dot displacement and the TMS-evoked potential. Next we wanted to study the neural correlate of surface segregation and to cancel out the neural effect of local dot displacement, the TMS-evoked potential, and signals related to figure border processing. We therefore subtracted ERPs on trials containing frame stimuli from ERPs on stack trials (Figure 4) for each rTMS condition separately. The resulting difference waves (stack-frame difference) now reflect surface segregation and no longer contain activity related to local dot displacement, the TMS-evoked potential, and figure border detection (Wokke et al., 2012; Scholte et al., 2008).
We performed random-effects analyses by applying sample-by-sample paired t tests (two-tailed) to test which samples of the subtractions differed significantly from zero. To reduce the amount of comparisons, we selected time windows that were identified in previous literature (Wokke et al., 2012; Pitts et al., 2011; Scholte et al., 2008; Caputo & Casco, 1999; Bach & Meigen, 1997) as relevant for figure border detection and surface segregation. We choose a time window between 80 and 230 msec after stimulus onset to statistically test relatively early differences related to figure border detection (in figure-homogenous subtractions, see above). To study the neural correlates of surface segregation, we choose a time window between 150 and 300 msec after stimulus onset to statistically test differences between ERPs on trials containing stack and trials containing frame stimuli. We used the mean of the figure-homogenous and stack-frame subtractions in the above-described time windows to directly test for differences between the rTMS conditions (see Figure 7).
EXPERIMENT 2 METHODS: BILATERAL SINGLE-PULSE TMS
Five undergraduate psychology students of the University of Amsterdam (four women, aged between 19 and 23 years) participated in this study for financial compensation (three participants also participated in Experiment 1). All participants had normal or corrected-to-normal vision and were naive to the purpose of the experiments. Participants were screened before the experiments (Rossi et al., 2009; Wassermann, 1998). Before the experiments, participants gave their written informed consent. The ethics committee of the Psychology department of the University of Amsterdam approved all procedures. The two “new” participants took part in a previous study performing the same task while stimulating the early visual cortex and recording EEG signals (∼2400 trials), making them well trained and accustomed to the experimental design.
The task set-up of this experiment was almost identical to that of the first experiment, therefore only the differences between Experiments 1 and 2 are described below.
In the first experiment, the stimuli were presented on the lower left side of the fixation dot. One might argue that it could be possible that the off-center positioning of the stimuli degraded the perception of the stimuli such that the differences between the stimuli did no longer accurately reflect differences in figure-ground segregation. To exclude this possibility, we therefore positioned the stimulus centrally in the second experiment instead of in the lower left side of the fixation dot (in this experiment, we bilaterally stimulated LO and V5/HMT+, see below). During Experiment 2, the monitor was placed at a distance of ∼120 cm in front of the participant (each centimeter now subtended a visual angle of 0.48°; we increased the distance to keep performance below near perfect). Because we increased the distance from the participant to the screen, the visual angle of the background (13.49°), the figure frame (2.42°), and the inner figure (1.81°; 24.8 cd/m2) changed accordingly.
In the second experiment, each participant performed a total of 12 experimental blocks (six blocks per TMS target location) in an average of four sessions, each containing 96 trials (creating a total of 48 trials per condition). In contrast to Experiment 1, we did not record EEG signals during Experiment 2.
Single-pulse TMS Protocol
We bilaterally stimulated areas V5/HMT+ and LO using single-pulse TMS. Therefore, we used an additional Magstim Rapid2 stimulator and a second 70-mm figure-of-eight coil. The coils were positioned using the same navigation system as described above (see rTMS Protocol). During the recording of each block, we were only able to track one coil (which was sufficient for this experiment, because head movement always led to displacements of the center of both coils relative to the target position). We used the same coil displacement criterion as during Experiment 1. About 90% of the phosphene threshold was used for stimulator output (mean = 63%, SD = 3.11% of max stimulator output). We simultaneously stimulated left and right LO or left and right V5/HMT+ in an early (100 msec), intermediate (160 msec), and late (240 msec) time window after stimulus presentation, intermixed with trials without TMS, thus creating a total of four TMS SOAs. The three TMS SOAs were based on a previous study using a similar task (Wokke et al., 2012).
On the behavioral data from the second experiment, we performed a repeated-measures ANOVA on mean percentage correct, with factors TMS SOA (no, early, intermediate, and late), Stimulus Type, and TMS Target Location. A repeated-measures ANOVA was also performed on mean RTs with factors TMS SOA (no, early, intermediate and late), Stimulus Type, and TMS Target Location. RTs of less than 100 and greater than 1500 msec were excluded from all analyses.
Behavioral Results: rTMS
To explore how dorsal and ventral cortical regions contribute to figure-ground segregation during a motion-defined figure discrimination task, we applied a train of five TMS pulses (prestimulus at 10 HZ) over either right V5/HMT+ or right LO while concurrently recording EEG signals. To study the behavioral effect of stimulation, a 3 (rTMS Condition: LO, V5/HMT+ and no rTMS) × 3 (Stimulus Type: stack, frame and homogenous) repeated-measures ANOVA on accuracy was performed. We observed a main effect of rTMS Condition (F(2, 12) = 5.27, p = .023) and, more interestingly, an interaction effect between rTMS Condition and Stimulus Type (F(4, 24) = 3.82, p = .015). To further study these effects, we performed a 3 (rTMS and no rTMS) × 3 (Stimulus Type: stack, frame and homogenous) repeated-measures ANOVAs on accuracy for each target location (LO and V5/HMT+) separately.
In the V5/HMT+ condition, we found no main effects of Stimulus Type (F(2, 12) = 1.67, p = .23) or rTMS (F(1, 6) = 2.02, p = .205) on performance scores. This means that, in the MT condition, performance scores did not differ between the three stimuli and that rTMS had no stimulus unspecific effect on performance scores. However, we did find a significant interaction effect (F(2, 12) = 12.29, p = .001) between Stimulus Type and rTMS. Depending on Stimulus Type, disruption of V5/HMT+ influenced performance scores. Post hoc paired t tests demonstrate (Figure 2A) that rTMS applied over V5/HMT+ significantly impaired performance but did so exclusively for stack stimuli (LSD-corrected, p = .012).
In the LO condition, we observed a main effect of rTMS (F(1, 6) = 26.71, p = .002) on performance scores (no main effect of Stimulus Type: F(2, 12) = 1.687, p = .226). We found a marginally significant interaction effect between Stimulus Type and rTMS (F(2, 12) = 3.65, p = .058). In contrast with the V5/HMT+ condition, accuracy increased after stimulation of LO (Figure 2B) for both stack (LSD-corrected, p = .023) and frame (LSD-corrected, p = .034) stimuli.
Results show that rTMS was able to influence performance scores when targeting right LO or right V5/HMT+. Surprisingly, stimulating LO or V5/HMT+ had opposing effects on task performance. Stimulation of V5/HMT+ resulted in decreased stack detection, without affecting overall performance (no significant main effect of rTMS), suggesting a bias shift (Figure 2A shows a decrease in stack detection and a small nonsignificant increase in frame detection after disruption of V5/HMT+). In contrast, performance increased when targeting LO and did so specifically for stack and frame stimuli (Figure 2B).
To study whether the observed effects were related to differences in the no-rTMS condition between the V5/HMT+ or LO condition, we performed repeated-measures ANOVAs for each target location; however, this time we collapsed the no-rTMS conditions across stimulation sites. In the V5/HMT+ condition, we still did not observe a main effect of Stimulus Type (F(2, 12) = 1.89, p = .19) or rTMS (F(1, 6) = 0.32, p = .59). However, we still observed a significant interaction effect between Stimulus Type and rTMS (F(2, 12) = 8.06, p = .006). In the LO condition, we again found a main effect of rTMS (F(1, 6) = 12.06, p = .01) and no main effect of Stimulus Type F(2, 12) = 1.52, p = .26). However, in the LO condition, we no longer observed the marginal significant interaction effect between Stimulus Type and rTMS (F(2, 12) = 1.90, p = .19). In addition, we directly compared performance scores for stack stimuli between the LO and V5/HMT+ TMS condition. Paired t tests (one-sided) revealed that performance scores were higher when LO was stimulated (t(6) = 3.01, p = .012) in comparison with performance scores in the V5/HMT+ condition. There was no difference when we compared performance scores for frames between the LO and V5/HMT+ condition (t(6) = 1.39, p = .11).
RT analysis showed a main effects of Stimulus Type (F(2, 12) = 8.72, p = .005) and rTMS Location (F(1, 6) = 5.98, p = .016). We observed a marginal significant interaction between Stimulus Type and rTMS Location (F(4, 24) = 2.213, p = .098). We studied the RT effects the same way as the accuracy scores; therefore, we performed a 3 (rTMS and no rTMS) × 3 (Stimulus Type: stack, frame and homogenous) repeated-measures ANOVA on RTs for each target location (LO and V5/HMT+) separately.
When stimulating right V5/HMT+ we did not observe a significant interaction effect between rTMS and Stimulus Type (F(2, 12) = 0.51, p = .613). However, we did find main effects of Stimulus Type (F(2, 12) = 4.92, p = .028) and rTMS (F(1, 6) = 28.68, p = .002). Post hoc paired t tests showed that rTMS applied over right V5/HMT+ slowed down participants unrelated to stimulus type (LSD-corrected, p < .001).
In the LO condition, we also found main effects of Stimulus Type (F(2, 12) = 9.86, p = .003) and rTMS (F(1, 6) = 12.85, p = .012). Again post hoc paired t tests showed that participants responded more slowly after stimulation (LSD-corrected, p = .012). In both the V5/HMT+ and LO conditions, participants responded more slowly to frame stimuli than to stack or homogenous stimuli (LSD-corrected, all ps < .036).
RT results show that rTMS influenced RTs similarly for both rTMS locations. Participants became slower when rTMS was applied over V5/HMT+ or over LO.
To rule out that our behavioral effects were caused by unspecific rTMS effects (e.g., cutaneous stimulation or noisy clicks), we added an extra control session, in which rTMS was applied over vertex. Results show that vertex stimulation did not influence performance scores. We did not find a main effect of rTMS (F(1, 5) = .095, p = .77) or Stimulus Type (F(2, 10) = 2.91, p = .101) or an interaction effect between Stimulus Type and rTMS (F(2, 10) = 0.93, p = .43). In contrast, analysis of RTs during vertex stimulation did reveal a significant main effect of rTMS (F(1, 5) = 22.1, p = .005). Participants became slower when stimulated over vertex than without rTMS (LSD-corrected, p = .005), yet unrelated to stimulus type (interaction effect between Stimulus Type and rTMS: F(2, 10) = 0.43, p = .66).
These findings show that the effects of rTMS on performance scores are most likely because of disruption of neural activity specific for LO and V5/HMT+. In contrast, RTs were influenced by rTMS unrelated to target location (vertex, right LO or right V5/HMT+). RT effects we found in this study are therefore most likely related to unspecific (nonneural) rTMS effects.
Behavioral Results: Single-pulse TMS
The above-described results demonstrate that disruption of dorsal area V5/HMT+ and ventral area LO influence motion-defined figure-ground segregation differentially. When we targeted V5/HMT+ we observed reduced performance scores selectively for stack stimuli, possibly because of a bias shift (see Figure 2A). Surprisingly, stimulation of right LO resulted in increased performance scores for both stack and frame stimuli.
To further study the effects found with rTMS, we conducted a second experiment. With a similar task set-up we now used bilateral single-pulse TMS to disrupt processing in LO and V5/HMT+. In this experiment, we briefly disrupted both areas at several time points after stimulus presentation. This design allowed us to investigate the temporal dynamics of the behavioral effects found with rTMS and study whether behavioral effects because of disruption of dorsal and ventral areas could be found in similar time windows. In addition, using a different TMS protocol allowed us to validate the surprising effect of performance enhancement found when disrupting area LO with rTMS.
We performed two repeated-measures ANOVAs (TMS location × TMS SOA × Stimulus Type) on accuracy and RT. On accuracy we found a significant TMS Location × TMS SOA × Stimulus Type interaction effect (F(6, 24) = 2.66, p = .04) and a significant main effect of Stimulus Type (F(2, 8) = 9.011, p = .009). To further study these effects, we performed repeated-measures ANOVAs (TMS SOA × Stimulus Type) on accuracy for each TMS target location separately.
In the V5/HMT+ condition, we found a significant main effect of Stimulus Type (F(2, 8) = 8.32, p = .01) and a marginally significant interaction effect between Stimulus Type and TMS SOA (F(6, 24) = 2.41, p = .06). Figure 3 shows that disruption of V5/HMT+ exclusively impaired performance on stack stimuli, specifically using an early SOA (early SOA vs. no TMS: LSD-corrected, p = .04; early vs. intermediate SOA: LSD-corrected, p = .06) and marginally significantly in a late time window (late vs. intermediate SOA: LSD-corrected, p = .06). Performance scores were lower for stack stimuli in comparison with frame and homogenous stimuli (stack vs. frame: LSD-corrected, p = .04; stack vs. homogenous: LSD-corrected, p = .04).
We found a main effect of Stimulus Type (F(2, 8) = 8.99, p = .009) and a main effect of TMS SOA (F(3, 12) = 9.97, p = .001) when LO was targeted. No TMS SOA × Stimulus Type interaction was found (F(6, 24) = 1.88, p = .13). In contrast to V5/HMT+ stimulation, post hoc tests showed that performance was enhanced specifically when using an early (early SOA vs. no TMS: LSD-corrected, p = .006; early vs. late SOA: LSD-corrected, p = .01) and intermediate TMS SOA (intermediate vs. late SOA: LSD-corrected, p = .02; see Figure 4). Although we did not test per stimulus type whether TMS timing influenced performance scores (because there was no TMS SOA × Stimulus Type interaction), Figure 4B seems to indicate that the main effect of TMS SOA is mainly driven by stack and frame stimuli and not so much by the homogenous stimulus.
As in the V5/HMT+ condition, performance scores were lowest for stack stimuli (stack vs. frame: LSD-corrected, p = .04; stack vs. homogenous: LSD-corrected, p = .03).
Similar to the rTMS experiment, we pooled the no-TMS conditions across stimulation sites to test whether the observed effects were not because of a difference in the no-TMS condition. In the V5/HMT+ condition, we again found a significant main effect of Stimulus Type (F(2, 8) = 10.41, p = .006) and a marginally significant interaction effect between Stimulus Type and TMS SOA (F(6, 24) = 2.17, p = .08). When LO was targeted, we found a main effect of Stimulus Type (F(2, 8) = 7.41, p = .02) and a main effect of TMS SOA (F(3, 12) = 8.37, p = .003). Again no-TMS SOA × Stimulus Type interaction was found (F(6, 24) = 1.20, p = .34). Paired t tests (one-sided) between performance scores on stacks in the LO and V5/HMT+ condition revealed that performance was higher when LO was stimulated compared with stimulation of V5/HMT+ in an early time window (t(4) = 5.61, p = .003). For frames we observed a similar difference in an early window when comparing both stimulation sites (t(4) = 2.41, p = .04).
RT analysis showed a significant TMS Location × TMS SOA interaction (F(3, 12) = 6.03, p = .008). Further analysis demonstrated a main effect of TMS SOA on RT in both the V5/HMT+ and LO condition (F(3, 12) > 6.83, p < .006). Post hoc tests demonstrated that participants responded more slowly when TMS was applied over V5/HMT+ using an early and intermediate SOA compared with no stimulation (early SOA vs. no TMS: LSD-corrected, p = .04; intermediate SOA vs. no TMS: LSD-corrected, p = .007). In the LO condition, however, responses were slowed down when we used a late TMS SOA (late vs. early SOA: LSD-corrected, p = .03; late SOA vs. no TMS: LSD-corrected, p = .003). These RT results seem to hint at a speed–accuracy effect (although usually speed-accuracy effects consist of speeded responses in combination with decreased performance). When V5/HMT+ was targeted, RTs became slower when participants' performance decreased (early TMS time window). When LO was targeted, participants became faster when performance increased (early TMS time window). These RT effects should however be met with caution. Previous findings (using sham stimulation in an almost identical task set-up) demonstrate that specifically RTs are highly susceptible to unspecific TMS effects (see Wokke et al., 2012, for single-pulse sham stimulation).
In this second experiment, we used single-pulse TMS to study the temporal dynamics of dorsal/ventral contributions during figure-ground segregation. Importantly, disruption of neural activity in an early time window (∼100 msec) mainly affected performance scores. Furthermore, these behavioral effects found in an early time window were found to be antagonistic for the V5/HMT+ and the LO condition. Performance decreased selectively for stack stimuli when stimulating V5/HMT+ in an early time window, whereas performance increased mainly in an early time window when targeting LO (Figures 3 and 4).
Effect of rTMS on Neural Correlates of Figure-ground Segregation
To determine the effect of disruption of LO and V5/HMT+ on previously found neural correlates of figure-ground segregation (Wokke et al., 2012; Scholte et al., 2008), we concurrently recorded EEG signals during the rTMS experiment (see Methods Experiment 1). We compared ERPs on figure trials with ERPs on homogenous trials for each rTMS condition separately (no rTMS, right LO, and right V5/HMT+). Figure 5A shows ERPs on figure and homogenous trials in the no-rTMS condition starting to deflect from each other around ∼150 msec (significant interval: 184–203 msec; t > 2.45, p < .05). Stimulation of right MT did not alter this figure-homogenous difference (significant interval: 184–207 msec; t > 2.45, p < .05; see Figure 5B). However, targeting right LO with rTMS seemed to enhance the difference between figure and homogenous signals (significant intervals: 98–141 and 184–215; t > 2.45, p < .05; see Figure 5C). However, to find out if there is a difference between the rTMS conditions, we have to compare the difference between figure and homogenous signals of the three rTMS conditions with each other. Direct comparison of the mean figure-homogenous difference (Figure 7A) revealed that this difference is greater in the LO condition than in the V5/HMT+ condition (t(6) = 2.52, p = .023, one-tailed).
To disentangle low-level (figure boundary detection) from higher-level (surface segregation) processes during figure-ground segregation, we subtracted ERPs on frame trials from ERPs on stack trials for each rTMS condition separately. These subtractions reflect the process of surface segregation, whereas signals related to border detection or local dot displacement are canceled out (see Methods). Figure 6A shows a significant deflection between responses evoked by stack and frame stimuli appearing around ∼190 msec (significant interval: 184–203 msec; t > 2.45, p < .05) in the no-rTMS condition. This significant stack-frame difference was abolished in the condition where we applied rTMS over right V5/HMT+ (Figure 6B), but still present in the LO condition (significant interval: 215–230 msec; t > 2.45, p < .05; Figure 6C). A direct comparison (Figure 7B) shows that rTMS over LO increases stack-frame differences in comparison with the no-rTMS condition (t(6) = 2.57, p = .022) and the V5/HMT+ condition (t(6) = 2.09, p = .041).
Disruption of activity in V5/HMT+ or LO seemed to specifically affect neural correlates of surface segregation (Figures 5, 6, and 7). Furthermore, in line with above-described behavioral results, disruption of activity in V5/HMT+ or LO had opposing effects on neural correlates of surface segregation (see Figures 7 and 8). Figure 6 suggests a decrease in signaling related to surface segregation after disruption of right V5/HMT+. However, no significant difference between the no-rTMS and the V5/HMT+ condition could be observed (Figure 7B). In contrast, Figure 7B shows that stimulation of right LO resulted in enhanced signaling related to surface segregation.
Motion and object (shape and surface) information are commonly thought to engage two functionally and anatomically separate processing streams (Milner & Goodale, 1992; Mishkin et al., 1983). Recently, a growing amount of studies argue against such a strict and absolute distinction (de Haan & Cowey, 2011; Schenk & McIntosh, 2010; Ellison & Cowey, 2009). In this study, we probed the role of (early) dorsal and ventral extrastriate regions by disrupting neural signaling in object selective area LO and motion sensitive area V5/HMT+ during a motion-defined figure discrimination task. In two separate experiments, we observed that stimulating LO or V5/HMT+ had opposing effects on task performance and neural correlates of figure-ground segregation. Disruption of V5/HMT+ resulted in decreased performance associated with higher-level processing steps during motion-defined figure-ground segregation. This effect was mainly found when TMS was applied in an early (100 msec) time window after stimulus presentation. In contrast, participants' performance improved when LO was targeted with TMS. This effect manifested itself also mainly when TMS was applied in an early time window (100 msec) after stimulus presentation. These findings suggest competitive interactions between dorsal and ventral extrastriate cortical areas.
Collaborative and Competitive Interactions between Dorsal and Ventral Areas
Although most agree with the general distinction of dorsal/ventral specializations, a growing amount of evidence suggests a more flexible segregation between both streams than current dominant models propose (Schenk, 2012; de Haan & Cowey, 2011; Ellison & Cowey, 2009; Doniger et al., 2002). The involvement of the ventral stream on typically dorsal tasks has been shown repeatedly (Schenk, 2012; McIntosh & Lashley, 2008). In these experiments, it has been demonstrated that “ventral” perceptual knowledge of an object contributes to the fine-tuning of the spatial programming of actions. Furthermore, Ellison and Cowey (2009) demonstrated that during a visuospatial distance discrimination task dorsal (posterior parietal cortex) and ventral (LO) regions processed information according to their own specialization but these regions cooperated to perform the task optimally.
A considerable amount of data in support of the dorsal/ventral segregation derives from patients with neurological disorders (Milner & Goodale, 2008). However, deeper analyses of typical “dorsal” or “ventral” neurological disorders (see Schenk, 2012; Pisella, Binkofski, Lasek, Toni, & Rossetti, 2006) argue against a strict dichotomy between “vision for perception” and “vision for action.” These findings advocate a network model (or patchwork model [de Haan & Cowey, 2011], but see Goodale & Milner, 2010) in which visual areas are being recruited depending on task demands or stimulus features. Recent findings showing competitive interactions between traditional dorsal and ventral systems seem to corroborate such a “pragmatic” network-oriented model. Jokisch and Jensen (2007) demonstrated that altering task demands (spatial vs. identification), such that the task engaged either the dorsal or ventral stream, resulted in inhibition of the dorsal stream when the task relied on ventral stream processing. In line with these findings, Walsh and colleagues (1998) induced task-specific impairments and enhancements by disrupting activity in motion sensitive area V5/HMT+. In their study, a series of six different visual search tasks (e.g., using motion, color, or form) demonstrated that mutual inhibition between dorsal and ventral visual regions could be induced. Depending on the visual property needed to perform the task, disruption of V5/HMT+ resulted in improved or impaired performance. It thus seems that, depending on the context in which information is being processed, different dorsal–ventral interactions exist.
In this study, we find additional evidence in support of such dorsal/ventral competitive interactions. Apparently, for motion-defined stimuli, ventral cortical areas are not contributing but rather “hindering” their proper processing by classically dorsal regions, and once freed from this “hinder,” these stimuli are processed more efficiently. It thus seems that ventral and dorsal systems compete when processing visual input, up to the point where dorsal regions can do their job more efficiently with ventral areas temporally lesioned than when in action.
Alternatively, stimulation of V5/HMT+ or LO could have had antagonistic neural effects. It could be that applying TMS over LO enhanced performance of this area or triggered a higher state of excitability in the associated ventral network whereas targeting V5/HMT+ resulted in a decreased state of excitability. However, in this study we used two different TMS protocols that have previously shown to disrupt both LO (Wokke, Vandenbroucke, Scholte, & Lamme, 2013; Koivisto, Railo, Revonsuo, Vanni, & Salminen-Vaparanta, 2011; Brighina et al., 2003) and V5/HMT+ (Théoret, Kobayashi, Ganis, Di Capua, & Pascual-Leone, 2002; Pascual-Leone & Walsh, 2001). It therefore seems unlikely that the previously demonstrated disruptive effects of both TMS protocols were reversed for LO in this study. Nonetheless, additional investigation should further elucidate the dynamics underlying the present results by zooming in on processing in dorsal/ventral cortical regions while ventral/dorsal areas are being disrupted (e.g., using a combination of fMRI and TMS). The methodology of this study unfortunately does not allow us to pinpoint the locus or direction of possible dorsal–ventral interaction, as well as the spread of the TMS-induced activity to dorsal or ventral regions when targeting ventral or dorsal regions, respectively. It could be that motion-sensitive brain regions feed information to shape/form specific regions in the ventral stream as Regan and colleagues suggested (Regan, Giaschi, Sharpe, & Hong, 1992). They observed that white matter lesions in parietal-temporal-occipital cortex resulted in the inability to perceive form from motion, without the loss of general motion perception. They attributed the inability to detect form from motion to damage to the interconnections between dorsal and ventral pathways that form a distributed functional network when performing a form from motion detection task (see also Ferber, Humphrey, & Vilis, 2003). In this study, however, the task did not so much depend on detecting form from motion but relied more on successful figure-ground segregation (see below), probably engaging a different (functional) network. Nonetheless, the observed effects in this study could derive from interactions between V5/HMT+ and LO, where competition and mutual inhibition shapes perception in the context of current task demands.
Figure-ground Segregation and Form from Motion
Traditionally, processing of the stimuli used in this study is thought to start with motion detection (local motion detectors in V1, V3, or MT), followed by the detection of motion-defined borders (opponent motion sensitive cells in MT or MST; Stoner & Albright, 1992; Movshon, Adelson, Gizzi, & Newsome, 1985). Next, mechanisms that are selective for motion-defined shapes are believed to play a crucial role (e.g., kinetic shape selective cells in IT; see Sary, Vogels, & Orban, 1993). However, in this study our stimuli were designed such that the stack and frame stimuli were identical with respect to the three stages of processing in detecting form from motion. When presenting a stack or frame stimulus, the average local motion was identical, both stimuli contained the same amount of motion discontinuities, and these motion discontinuities defined the same rectangle shapes. In contrast, the way filling-in processes progressed over the rectangle shapes was different for stack and frame stimuli: In frame stimuli the interior rectangle was “grouped” with the background, creating a frame standing out as a different surface. However, for the stack stimulus the interior rectangle filled in as a separate surface. Previous findings demonstrate that the three stages in processing form from motion are mediated by rapid feedforward sweep processing (see Lamme & Roelfsema, 2000) whereas filling-in is associated with reentrant processing mediated by horizontal and feedback connections (Wokke et al., 2012; Lamme, 1995). It therefore seems that form from motion processing precedes filling-in. However, in this study we used stimuli (stack and frame), which from the perspective of motion defined shape (form from motion) mechanisms are comparable. The observed effects of disrupting V5/HMT+ and LO are therefore probably more related to (motion-defined) figure-ground segregation than to processing form/shape from motion.
The task in this study probably engaged the dorsal stream more than the ventral stream, as the task required motion-defined figure-ground segregation and did not rely on shape processing per se. As a next step, it would be interesting to find out if the observed behavioral effects would reverse when task demands are shifted such that the task relied more on ventral than on dorsal stream processing (e.g., shape discrimination; see Ferber et al., 2003).
Neural Correlates of Figure-ground Segregation
In this study, we used stimuli that made it possible to differentiate between different levels of figure-ground segregation (Scholte et al., 2008; Vandenbroucke et al., 2008; Heinen et al., 2005). It has been well established that neural correlates of early stages of figure-ground segregation (such as figure boundary detection) can be found in early visual cortex (Lamme, 1995). However, previous studies have shown that figure-ground manipulations are also able to influence relatively late perioccipital ERPs in human EEG recordings (Wokke et al., 2012; Caputo & Casco, 1999; Bach & Meigen, 1997; Lamme, Van Dijk, & Spekreijse, 1992). For instance, by probing different regions of the classical face–vase figure (face, vase, or borders in between) or manipulating the amount of figure surface ERPs related to sequential stages in figure-ground segregation have been observed (Wokke et al., 2012; Pitts et al., 2011; Scholte et al., 2008). These studies show an early difference in ERPs related to figure border detection and a later occurring difference likely reflecting border ownership coding and/or surface segregation. In a previous study (Wokke et al., 2012), we observed that applying TMS over early visual cortex at different moments in time resulted in disruption of both figure border detection and later occurring surface segregation, demonstrating a causal relationship between neural activity in early visual cortex and figure surface segregation. Here results demonstrate that specifically these later occurring processes (surface segregation) were affected when we applied rTMS over V5/HMT+ (Figures 2 and 4).
The impact disrupting activity in LO had on neural correlates of figure-ground segregation manifested itself in an increased difference when comparing signals on stack trials with signals on frame trials (Figure 7B). Although the deflection of signals on figure trials from signals on homogenous trials also seem to be enhanced when targeting LO with rTMS, there was no significant difference between the no-rTMS and LO conditions (Figure 7A).
When we targeted V5/HMT+ there was no behavioral effect on frame detection, this might have contributed to the reason why we did not observe a difference between the no-rTMS condition and the V5/HMT+ condition when comparing the subtractions of signals on figure trials (stack and frame trials collapsed) and signals on homogenous trials (Figure 7A). In contrast, stack detection did deteriorate when targeting V5/HMT+, although this did not result in a significant decrease of the difference between signals on stack trials and signals on frame trials (Figure 7B).
The present findings suggest that early processes related to figure border detection are less dependent on areas beyond early visual cortex (see Figure 8 for effects on frames when applying TMS over V5/HMT+). In contrast, V5/HMT+ causally contributes to later emerging processes related to surface segregation, possibly by means of recurrent interactions with early visual cortex (Wokke et al., 2012; Pascual-Leone & Walsh, 2001; Zhou, Friedman, & Von Der Heydt, 2000; Lamme, 1995).
In summary, our findings support recently developed theories (Schenk, 2012; de Haan & Cowey, 2011), in which the segregation of the two main visual processing streams is considered not absolute, but rather flexible. In such a view, extrastriate areas are being recruited or inhibited depending on the stimulus category and current task demands.
We thank Tinka Beemsterboer for her help with data acquisition and Dick Schwafl for his useful comments.
Reprint requests should be sent to Martijn E. Wokke, Amsterdam Brain & Cognition Center, University of Amsterdam, DS.08, Nieuwe Achtergracht 129-131, 1018XA, Amsterdam, NH, Netherlands, or via e-mail: firstname.lastname@example.org.