In two EEG experiments, we studied the role of visual attention during the preparation of manual movements around an obstacle. Participants performed rapid hand movements to a goal position avoiding a central obstacle either on the left or right side, depending on the pitch of the acoustical go signal. We used a dot probe paradigm to analyze the deployment of spatial attention in the visual field during the motor preparation. Briefly after the go signal but still before the hand movement actually started, a visual transient was flashed either on the planned pathway of the hand (congruent trials) or on the opposite, movement-irrelevant side (incongruent trials). The P1/N1 components that were evoked by the onset of the dot probe were enhanced in congruent trials where the visual transient was presented on the planned path of the hand. The results indicate that, during movement preparation, attention is allocated selectively to the planned trajectory the hand is going to take around the obstacle.
When we act in complex environments, the avoidance of obstacles is often a key problem of movement programming. Whether we reach out for the salt that is on a crowded table right behind a full glass of water or whether we grasp the spoon that lies close to the cup of hot tea, our goal-directed movements are often at risk to interfere with other objects that we need to avoid (Chapman & Goodale, 2008, 2010a, 2010b; Striemer, Chapman, & Goodale, 2009; Aivar, Brenner, & Smeets, 2008; Cameron, Franks, Enns, & Chua, 2007; Liu & Todorov, 2007; Welsh & Elliott, 2004; Hamilton & Wolpert, 2002; Sabes, Jordan, & Wolpert, 1998; Sabes & Jordan, 1997). Although adults avoid obstacles with apparent ease, such coordinate movements represent a difficult task, which necessitates several years of training—as can be observed in the development of infants and children (e.g., Kretch & Adolph, 2017; Agyei, van der Weel, & Van der Meer, 2016; Juett & Kuipers, 2016).
The elaborateness of implementing such a task can also be appreciated from a point of view of robotics engineering (Ben-Ari & Mondada, 2018; Minguez, Lamiraux, & Laumond, 2016). Here, two prominent accounts have been put forward. The first one implements the avoidance of an obstacle by so-called “potential fields,” which either attract or repulse the robotic effector (see, e.g., Kim, Lee, Oh, Kang, & Kang, 2017; Zhou & Li, 2014). The optimal path, along which the effector will go around an obstacle and reach the final goal, is a dynamic combination of the repulsion from the obstacle and the attraction to the goal location (Zhou & Li, 2014). A second and biologically more plausible account uses intermediate goals, so-called milestones to compute a route around an obstacle (e.g., Chiang, Malone, Lesser, Oishi, & Tapia, 2015). Here, the space of interest is likened to a roadmap (path-guided), which is first subclassified into “forbidden regions,” that is, the obstacle(s), and “free space” (everywhere else except the obstacles). If the obstacle is in-between the starting point and the movement goal, potential intermediate goals are constructed in the free space and again the algorithm tests for possible collisions (Minguez et al., 2016). Here, multiple alternative routes through various potential intermediate goals can be compared, and one is finally selected as the actual path. One critical problem in this account is that the best path remains underdetermined. To find the optimal path, complex functions have to be integrated along all possible trajectories, and the outcomes have to be compared—a computational effort that takes time.
This problem is also well known in human movement science, mostly referred to as the problem of motor equivalence (Bernstein, 1967; Lashley, 1930): Every goal position can be reached through an infinite number of ways (path underdetermination). So far it remains unknown how the brain chooses a particular movement path—say for the hand reaching around an obstacle—from all the possible alternatives. This is not only true for a given effector: A certain motor output can also be achieved by different effectors, as has been prominently illustrated in the case of cursive handwritings (e.g., Meulenbroek, Rosenbaum, Thomassen, Loukopoulos, & Vaughan, 1996; Wright, 1993; Lashley, 1942). A possible solution to the problem of motor equivalence has been formulated by proposing a simple model that could simulate motor equivalent movements (Meulenbroek et al., 1996; see also Van der Wel, Fleckenstein, Jax, & Rosenbaum, 2007). According to this model, many complex movements can be described as a sequence of movement primitives, each of which may have an intermediate goal itself. The authors argue that actors may have access to abstract spatiotemporal forms, such as the repetitive succession of intermediate goals, and that these spatiotemporal forms allow to produce similar motor output with different effectors as well as to scale the output to different absolute sizes (see also, e.g., Keele, Cohen, & Ivry, 1990). Van der Wel and collaborators, for example, (Van der Wel et al., 2007) demonstrated that, also in manual obstacle avoidance, such abstract spatiotemporal forms play an important role: Their data suggest that trajectories around an obstacle can be primed by previous movements (see also Jax & Rosenbaum, 2007) and that these priming effects generalize over the workspace and scale with the height of the obstacle. Here, the spatiotemporal form of the movements around an obstacle could be either the overall shape of the trajectory or a series of individual points, for example, the points of maximal excursion or the points of maximal velocity—as it is the case in handwriting movements. Also some recent work by Wada and Kawato proposed that spatial intermediate goals could be used to aid the formation of complex curved trajectories (Wada & Kawato, 2004). The combination of intermediate goals along the reach trajectory and a final goal could therefore be a biologically plausible way to choose an efficient path around an obstacle—similar to what robots do.
What does this mean for visual perception? As many studies have demonstrated, we do not process all incoming visual information to the same extent but filter out irrelevant parts to select and process in depth only those parts of the visual scene that are relevant to our behavioral goals. In terms of movement preparation, it has been shown, for example, that attention preselects the goal location for an intended eye movement (Deubel & Schneider, 1996; Kowler, Anderson, Dosher, & Blaser, 1995). Furthermore, visual attention also serves the preparation of goal-directed manual reaches (Baldauf, Wolf, & Deubel, 2006).
Moreover, the selection of goals for upcoming movements is not restricted to single locations. It has been shown that multiple movement goals were selected in parallel before movement onset if several movements were planned in rapid succession (Baldauf, Cui, & Andersen, 2008; Baldauf & Deubel, 2008a, 2008b; Baldauf et al., 2006). In such cases, the attentional spotlight splits into multiple, spatially distinct foci (Baldauf & Deubel, 2008a, 2008b), forming an “attentional landscape” (Baldauf, 2011; Baldauf & Deubel, 2010). Some rare experiments also studied the distribution of visual attention in situations in which obstacles had to be avoided by manual reaches to a target (Deubel & Schneider, 2005). Here again, covert attention was not only restricted to the goal of the reach, but the obstacle was coselected as well—before the movement was initialized. In another landmark study, Johansson and coworkers tracked open gaze behavior while participants manually transported an object around an obstacle. They found that participants often fixated the obstacle before and during movement execution (Johansson, Westling, Bäckström, & Flanagan, 2001). In some trials, they even observed eye fixations to fall outside the obstacle, in the open space between the obstacle's edge and the manipulandum. One conclusion that can be drawn from all these instances is that the visual resources are flexibly adjusted to the actual behavioral needs, that is, to whatever is relevant for the preparation of an upcoming movement, including areas with potential obstacles.
Do actors also covertly attend to the path they choose around an obstacle? Given that invariant spatiotemporal forms may play a role for the programming of the reach around an obstacle, relevant positions may be visually selected in advance before piloting through them. Parts of the intended reach trajectory could function as intermediate goals and may be coselected by visual attention, similarly as immediate and subsequent goals are selected in reach sequences (Baldauf & Deubel, 2008a, 2010; Baldauf et al., 2006, 2008).
In the present ERP study, we tested with a dot probe paradigm the hypothesis that visual attention also selects locations along the upcoming path around an obstacle. Particularly, we were interested in the distribution of visual attention in the field during the preparation of obstacle avoidance reaches. We analyzed the amplitude of the neural response (ERP) elicited by the onset of a task-irrelevant probe stimulus to infer how much processing resources were allocated to a certain location. The visual ERP has previously been shown to be particularly sensitive to the direction of spatial attention: Probe stimuli that were flashed at attended locations elicit bigger sensory-evoked P1/N1 components than identical stimuli at unattended locations (Bagherzadeh, Baldauf, Lu, Pantazis, & Desimone, 2017; Baldauf & Desimone, 2016; Baldauf, Grossman, Hu, Bayden, & Desomine, 2016; Baldauf, 2015; Baldauf & Deubel, 2009; Hillyard, Mangun, Woldorff, & Luck, 1995; Luck et al., 1994; Mangun, Hillyard, & Luck, 1993).
EXPERIMENT 1: CONGRUENT VERSUS INCONGRUENT PATH
Eleven students, aged between 24 and 32 years (five women) participated in the experiment. They all had normal or corrected-to-normal vision and were right-handed. They were paid for their participation and gave their informed consent in advance of the experiments.
Figure 1A provides a sketch of the experimental setup. The participants sat in a dark room. The visual stimuli were presented on a 21-in. monitor (100 Hz frame frequency, 1024 × 768 pixels resolution). The active screen size was 40 × 30 cm at a viewing distance of 58 cm. The hand movements were performed on a slightly inclined working plane in front of the participant. A half-translucent mirror was placed in front of the participant such that the visual stimuli appeared to be projected onto the working plane. This mirror allowed hand movements without visual feedback about the position of the hand. Also, the use of the mirror avoided parts of the visual scene being obstructed by the participant's hand. All visual stimuli had a luminance of 23 cd/m2 and were presented on a gray background with a mean luminance of 2.2 cd/m2. A central loudspeaker in the back of the participant delivered the auditory cues.
A Fastrak electromagnetic position and orientation measuring system (Polhemus, Inc., 1993) continuously recorded the movements of the right hand. The system consists of a sender unit and a small receiver mounted on the tips of both index fingers. The sender unit was placed at a distance of 60 cm from the participant. The device had a spatial accuracy of 0.03 in. and a frequency bandwidth of 120 Hz. The signal delay was approximately 4 msec. To provide visual feedback about the hand position during an initial positioning period as well as after execution of the reach a small LED (5 mm in diameter) was attached to the sensor and controlled by the computer. Central eye fixation was controlled by EOG. An adjustable chin rest helped reducing movement artifacts.
Stimuli and Procedure
Figure 1B shows the succession of stimuli in a typical trial of Experiment 1. The screen continuously displayed a starting box (at midline of the lower workspace with 10° eccentricity), in which the right index finger had to be placed at the very beginning of each trial, the margin of a static obstacle in the center, and a target cross in the upper half of the screen. Furthermore, a small fixation cross was provided in the center of the obstacle itself. Spatially congruent with the visual outline of the obstacle, a real wooden cylinder was positioned in the pointing plane underneath the mirror. The cylinder had a diameter of 8 cm and was 2 cm in height. The relatively small height of the obstacle allowed the participants to comfortably reach around on both sides.
When the trial started participants used the illuminated LED on the tip of their index finger to position their hand in the starting box. Then the LED was extinguished, and after a random interval of 600–900 msec, an acoustical go signal was presented, which had a frequency of either 500 or 200 Hz. The participants had to reach with their right hand around the obstacle to the target as soon as they heard the go signal. If the go signal was a low-frequency beep, the participants had to avoid the central obstacle on one side; if the go signal was a high-frequency beep, they reached the target on a route along the other side. The mapping of low versus high pitch tone to cue left versus right movement paths was counterbalanced. Participants were instructed to slide their fingertip around the obstacle and to keep contact between their index finger and the working plane during the reach, restricting the obstacle avoidance reaches to the 2-D space. The participants were asked to reach for the target as fast and as accurately as possible. Strong emphasis was placed on never touching the obstacle. After the execution of the required movement, participants received visual feedback about their pointing accuracy in reaching target.
To measure the deployment of visual attention during movement preparation, a task-irrelevant dot probe was flashed 150 msec after the acoustic go signal—well before the instructed reach actually started. The dot probe was a circular white disk with a diameter of 1.2° and appeared for 70 msec either to the right or to the left of the obstacle with an eccentricity of 8°. The lateral distance from the object was adjusted in pretests to ensure that the dot probe was flashed close to the actual trajectories the participants would take. Therefore, each participant performed a pretest block of 100 trials without EEG recording. This pretest revealed that most participants choose a very similar path around the obstacle and pass the cylinder in a distance of, on average, 8.4°.
The participants performed four experimental blocks, each consisting of 100 trials. The go signal instructed to reach either on the left or right side around the obstacle along the respective pathway. The critical factor that was varied in this experiment was the position of the dot probe relative to the instructed movement path: (1) in congruent trials the dot probe appeared on the side on which the hand was planned to move along and (2) in incongruent trials, however, the dot probe was flashed on the opposite, movement-irrelevant side. In total, this led to four different experimental conditions (2 pathways × 2 dot probe positions). The conditions were randomized in each trial, and each condition was repeated 25 times in each block.
We used a BrainAmp system (Brain Products) to continuously record from 64 Ag/AgCl electrodes mounted in an elastic cap (EasyCap, FMS). The electrodes were placed according to the international 10–10 system and referenced to Cz. The horizontal electrooculogram was bipolarly recorded from the outer canthi of both eyes. The vertical electrooculogram was recorded from electrodes above and below the left eye. All electrode impedances were kept as equal as possible and below 5 kΩ. The signals were amplified and filtered online by a 0.1–100 Hz bandpass filter. The digitalization rate was 500 Hz. The recorded signals were then 40 Hz low-pass filtered offline.
The continuous EEG data were epoched into analysis windows of 600-msec duration, starting 100 msec before and ending 500 msec after the presentation of the go signal. Trials with eye blinks (a voltage at Fpz exceeding ±80 μV), saccadic eye movements (a voltage at horizontal EOG or vertical EOG exceeding ±80 μV), or with muscle artifacts (a voltage at any site exceeding ±100 μV) were excluded from further analysis.
We computed separated ERP averages for the two relative positions of the dot probe with respect to the instructed pathway (congruent vs. incongruent), irrespectively of the hemifield, in which the dot probe had been flashed. All epochs were time-locked to the onset of the dot probe. The averages were computed relative to the 100-msec baseline before the onset of the go signal.
We calculated separate repeated-measures ANOVAs to analyze the mean amplitudes of the P1 and N1 components that were elicited by the dot probes appearing at the two relative positions. The two factors of these ANOVAs were Probe position (congruent vs. incongruent) and Electrode site with the levels O1, O2, P3, P4, C3, and C4. Statistical analyses were done with MATLAB and the R statistical package (Ihaka & Gentleman, 1996).
Rejection of Trials
A total of 5.4% of all trials were rejected either because eye movements occurred or because of other artifacts in the EEG recordings (e.g., muscle activity). Because we wanted to ensure that the dot probe was presented during the movement preparation period, we also excluded 3% of the trials with latencies shorter than 220 (150 msec SOA + 70 msec presentation time of the dot probe) and 7% of trials with movement onset latencies longer than 600 msec.
Figure 2A gives an example of some typical trajectories around the obstacle for one participant. The mean spatial error between the final landing position and the center of the target was 1.08°. Interestingly, the trajectories are rather similar across trials. The participants efficiently chose an optimal path for the reaching hand, which stayed remarkably constant.
Figures 2B and 2C show the latencies of movements along the left versus right path. When reaching along the left path, the hand started, on average, 392 msec (SE = 32 msec) after onset of the go signal. For reaches along the right path, mean latency was, on average, 402 msec (SE = 35 msec). The reaching movements took, on average, 284 msec (SE = 38 msec) along the left and 350 msec (SE = 41 msec) on the right path. The difference in mean latency between the left- and right-ward paths may be due to the fact that reaches along the left-hand side were slightly more difficult from a motor point of view (as reaches along both sides were executed with the dominant, i.e., right hand). Because the task-irrelevant visual transient served as a measure of the distribution of visual attention, it should not have affected the motor task such that, for example, its appearance on the planned trajectory hampers or delays the movement that is about to be programmed. Therefore, we analyzed whether the movement latencies depended on the relative probe position. A one-way ANOVA showed no significant main effect of the Probe position on the latency of the movement onset (p > .4). The movement latencies did not differ significantly between those trials, in which the probe was on the planned trajectory, and trials, in which the dot probe was presented on the opposite, movement-irrelevant side.
After initialization, the hand moved in a homogenously bended trajectory around the obstacle. The point of maximum excursion was reached after about 142 msec on about the same level with the obstacle, but with a horizontal offset of about 9.4° (9.6° on the left and 9.1° on the right-hand side). At this point the trajectories also had the minimum curvature. This “via point” was spatially congruent with the point of maximum velocity.
We analyzed the ERPs that were evoked by the flashing of the visual transient before movement onset to test whether critical parts of the planned path are visually selected in advance. The ERPs were collapsed across both hemifields in which they appeared, but contrasted on basis of their relative position in relation to the planned movement route (dot probe on the congruent vs. incongruent side). Figure 3A depicts the grand-averaged ERPs that were elicited at Oz by dot probes that either lay on the planned path of the hand (congruent condition, solid line) or lay on the opposite, movement-irrelevant side of the obstacle (incongruent condition, dashed line). A topography of the evoked response is provided in Figure 3B. Both ERPs were characterized by prominent P1 and N1 components in response to the flashed probe. The P1 component peaked 126 msec after onset of the dot probe; the N1 component reached its maximum about 182 msec after the probe. Both the amplitude of the P1 and N1 components were enhanced if the visual transient was flashed on the planned movement path as compared with the movement-irrelevant side. Figure 4 provides an overview of the elicited components at other electrode sites. The evoked components were most prominent at occipital (and occipitoparietal) sites (e.g., O1, O2, PO7, PO8, PO3, PO4, P3, P4) and decreased systematically in amplitude at more anterior sites (e.g., C3, C4). Also, the observed modulation of the P1 and N1 components, that is, the difference in amplitudes between the congruent and incongruent condition, was more pronounced at the posterior sites.
Statistical analyses further affirmed the modulation of the ERP components. For quantitative comparison of the components, we extracted the mean voltages of the P1 and N1 amplitudes in a time window of 30 msec around the respective peak. For the P1 component, this interval ranged from 111 to 141 msec (peak of the P1 at 126 ± 15 msec), and for the N1 component, the interval was from 167 to 197 msec after probe onset. For both data sets, we computed a two-way ANOVA. Here, the first factor was the Relative probe position with the levels congruent and incongruent. The second factor coded the Electrode site and had the levels O1, O2, PO7, PO8, PO3, PO4, P3, and P4. The ANOVA for the N1 component revealed a significant main effect of both factors without significant interaction (ME probe position: F(0.613, 6.130) = 9.10, p = .030, Greenhouse–Geisser [GG]-corrected; ME electrode site: F(4.291, 41.91) = 8.63, p = .001, GG-corrected; Probe Position × Electrode Site: F(4.291, 41.91) = 2.21, p = .0797, GG-corrected). For the smaller P1 component, the main effect of the Relative probe position on the amplitude did not reach significance (F(0.67, 6.70) = 3.34, p = .116, GG-corrected), but the main effect of Electrode side did (F(4.69, 46.9) = 4.10, p = .0042, GG-corrected).
In Experiment 1, we used a dot probe paradigm to probe the distribution of visual attention in the scene while participants prepared to reach around a central obstacle. The results showed that, during the preparation period of these movements, the visual system preferentially processes visual input from the movement-relevant side of the obstacle as compared with input from the movement-irrelevant side. In this first experiment, the probe was well outside the obstacle, and therefore the results suggest that visual attention is not only directed to the goal or the obstacle itself. Also other parts of the scene that are of prior interest during the planning stage, namely, the future hand path itself, are preselected in advance.
EXPERIMENT 2: THE OBSTACLE'S MARGIN
Experiment 1 showed that the side along which the participant is going to reach around an obstacle is preferentially processed during movement preparation. In advance of the movement initialization, attentional resources are allocated to the movement-congruent side, resulting in enhanced visual ERP components in response to movement-irrelevant probe stimuli presented there. We interpret this as the result of an attentional bias toward the future path, along which the hand is planned to travel. But what is it about the obstacle itself? Previous studies reported the attentional selection of hindering objects during movement preparation. For example, Johansson and colleagues (2001) found that an obstacle was fixated in about 80% of trials when actors maneuvered around it. Furthermore, Deubel and Schneider (2005) showed by a secondary discrimination task that attention is covertly deployed to a central obstacle immediately before reach initialization. In many attentional tasks, the visual object has been shown to be the unit of attentional selection (Zhang, Mlynaryk, Japee, & Ungerleider, 2017; Baldauf, 2015; Baldauf & Desimone, 2014; O'Craven, Downing, & Kanwisher, 2000). Therefore, in Experiment 2, we will directly compare the attentional facilitation on the future hand path with the attentional facilitation on the obstacle itself.
The direct quantitative comparison will provide a clue about whether the observed bias toward the future reach path, as observed in Experiment 1, is just a by-product of attention spreading out from the obstacle object or relevant parts of it, such as the object's margin, and coactivating the path locations to some extent, or whether the future hand path is indeed attended by itself and to a quantitative similar extent as the obstacle itself.
Participants, Experimental Stimuli, and Procedure
Eleven students, aged between 25 and 29 years (four women), participated in Experiment 2. They had normal or corrected-to-normal vision and were right-handed. Figure 5A illustrates the modified experimental stimuli in Experiment 2. The participants now had to fixate their eyes on the cross that marked the goal location on the upper end of the display. In the lower visual field was a starting box, from which the right index finger had to start the reach. Furthermore, the margin of a central obstacle was continuously displayed, which was congruent with the same 3-D obstacle as in Experiment 1 underneath the mirror setup.
When the acoustical go signal was presented (500 or 200 Hz), the participants had to reach with their right hand along the respective route around the obstacle to the target. The mapping of low/high pitch tone to cue the left/right movement path was counterbalanced. Again speeded responses were required, and it was emphasized never to touch the obstacle itself. After the movement was finished, visual feedback about the pointing accuracy was given.
Again, a task-irrelevant visual transient was flashed for 70 msec (SOA 150 msec) during movement preparation, either (1) on the side where the hand was instructed to reach along or (2) on the margin of the obstacle that faced to the instructed reach path, or symmetrically (3) on the other incongruent side. The metrics of the dot probe were kept the same. All dot probes were equidistantly presented with an extrafoveal eccentricity of 8°. The lateral distance from the object was adjusted to coincide with the reach paths as revealed in the pretests of Experiment 1. Eyes had to be fixated on the target location above the obstacle to allow for the possible dot probe locations to be equidistantly arranged.
The participants performed five experimental blocks, each consisting of 120 trials. As in Experiment 1, the go signal instructed to reach around the obstacle either on the left or right side. Figure 5A illustrates the relative positioning of the visual dot probe relative to the obstacle. The dot probe appeared either at the via point on the same side as the planned trajectory (congruent condition) or on the via point of the opposite, movement-irrelevant side (incongruent condition), or on the margin of the obstacle itself on the movement relevant side. In total, this led to six different conditions (2 pathways × 3 dot probe positions). The conditions were randomized in each trial and each condition was repeated 20 times in each block.
Rejection of Trials
A total of 8.0% of all trials were rejected because eye movement occurred or because of other artifacts in the EEG (e.g., muscle activity). As in the first experiment, we also excluded 4% of the trials with latencies shorter than 220 (150 msec SOA + 70 msec presentation time of the dot probe) and 8% of trials with movement onset latencies longer than 600 msec.
In Experiment 2, reaches along the left path had an average latency of 386 msec (SE = 37 msec) and took, on average, 291 msec (SE = 39 msec). Reaches along the right side had a mean latency of 395 msec (SE = 40 msec) and average reach duration of 310 (SE = 31 msec).
We analyzed the ERPs in response to the visual transient to probe the distribution of visual attention in the workspace. ERPs were collapsed across both hemispheres and contrasted on basis of their relative position in relation to the planned movement. Figure 5B shows the grand-averaged ERPs at a parieto-occipital site (PO8), elicited by visual transients on the planned path of the hand (congruent condition, solid line) or on the opposite, movement-irrelevant side of the obstacle (incongruent condition, dashed line), or on the margin of the object itself (congruent margin, dotted line). The P1 peaked 125 msec after onset of the dot probe; the N1 peaked at about 175 msec after the probe onset. Particularly, the amplitude of the N1 component was enhanced if the visual transient was flashed on the via point position of the planned route for the hand compared with the movement-irrelevant side. Interestingly, also the margin of the object itself seems to be preferentially processed, with visual transients presented there also eliciting an enhanced N1 component. Again, the evoked components were most prominent at occipital (and parieto-occipital) sites (e.g., O1, O2, PO7, PO8, PO3, PO4) and decrease in amplitude at more anterior sites.
We extracted the mean voltages of the more prominent N1 component in a time window of 30 msec around the respective peak. For the N1, the interval was from 150 to 180 msec after probe onset (peak of the N1 at 165 ± 15 msec). We then computed a two-way ANOVA with the first factor being the Relative probe position with the levels congruent, incongruent, and margin. The second factor coded the Electrode site and had the levels O1, O2, PO7, PO8, PO3, and PO4. The ANOVA for the N1 component revealed a significant main effect of both factors without any significant interaction (ME probe position: F(1.16, 11.60) = 8.2698, p = .0121, GG-corrected; ME electrode site: F(2.9, 29.0) = 3.6972, p = .0239, GG-corrected; Probe Position × Electrode Site: F(5.80, 58.0) = 0.9972, 0.4345, GG-corrected).
To quantitatively compare the extent to which visual attention was selectively deployed to either of the three positions of interest, we computed planned contrasts between the mean N1 amplitudes in the three attentional conditions. The evoked responses elicited by probes on the relevant path and at the obstacle's margin were both enhanced compared with the irrelevant side (t(10) = 3.1030, p < .012 and t(10) = 2.6982, p < .023, respectively; see Figure 5B) and to about the same level, that is, there was no significant difference between conditions congruent path and congruent margin: t(10) = −2.0517, p > .05.
The results of the second experiment mean that both the future path, that is, the planned trajectory, and the obstacle's margin were attended to the same extent, confirming that the selection of the future hand path (as already observed in Experiment 1) was not just a side effect of the visual selection of relevant object parts of the obstacle itself (spreading or falloff), but rather that the chosen route around the obstacle was selected itself and to about the same extent as the obstacle. Furthermore, this shows that, during movement preparation, attention is not only deployed to objects themselves (purely object-based attention) but also to important spatial locations along the path (spatial selection). It has been shown that, in complex movement tasks, multiple movement goals can indeed be selected in parallel well before the movement, establishing an “attentional landscape” of the workspace (Baldauf, 2011; Baldauf & Deubel, 2010) potentially even with the attentional spotlight being split into several foci (Gallivan et al., 2011; Chapman et al., 2010; Baldauf & Deubel, 2008a, 2008b).
In real-world interactions with the environment, we rarely act on isolated objects. More often, we have to coordinate our motor behavior in crowded workspaces. As a consequence, not only the final goal location of a reach is important to consider but also nontarget objects will have an influence on the attentional preparation of any reach or grasp. In cluttered scenes full of objects, the complete arrangement of the workspace will have to be taken into consideration.
Especially the presence of any obstacles poses important constraints on the ongoing preparation of goal-directed motor behavior (see, e.g., Tresilian, 1998, 1999, 2005; Tipper, Meegan, & Howard, 2002; Mon-Williams, Tresilian, Coppard, & Carson, 2001; Howard & Tipper, 1997; Tipper, Howard, & Jackson, 1997). Previous research has shown that the latencies of reach-to-grasp movements are prolonged if an obstacle is present as compared with the same movement executed without any obstacles (Biegstraaten, Smeets, & Brenner, 2003; Mon-Williams et al., 2001; Mon-Williams & McIntosh, 2000; see also Chapman & Goodale, 2010a), providing first experimental evidence for the additional need of anticipatory programming.
Our present results are very much in line with this general view. In Experiment 1, we showed that the side, along which the participant is going to reach around an obstacle, is preferentially processed during movement preparation. In advance of the movement initialization, attentional resources are allocated to the movement-congruent side, resulting in enhanced visual ERP components in response to movement-irrelevant probe stimuli there. This attentional facilitation was observed in Experiment 1 at the point of the intended trajectory, where the trajectory reaches its maximum excursion and, at the same time, its minimum curvature.
Two alternative mechanisms could have caused this first result. One could imagine, for example, that the obstacle primarily received full attention and that the programming of the avoidance path just biased the attention toward the movement-relevant side. More concretely, actors could, for example, specifically have attended to the relevant-sided margin of the obstacle. Therefore, the facilitated processing on the planned path would have been the result of visual attention asymmetrically radiating from the obstacle. From this perspective, we would assume that attentional effects are strongest at the obstacle itself and gradually decrease with increasing distance from the obstacle, and such an attentional gradient could have been biased toward the movement-relevant side of the configuration.
An alternative mechanism for the observed facilitation on the future reach path in Experiment 1 could be that the brain actively programs the entire trajectories or that control policies are selected, for which locations on the intended movement path are important (see, e.g., Christopoulos, Bonaiuto, & Andersen, 2015; Christopoulos & Schrater, 2015). Following this second mechanism, one would predict that certain positions of the path around an obstacle are preselected themselves, not just as a result from a gradually falling off spread of visual attention from the obstacle.
In Experiment 2, we therefore refined our attentional probing paradigm and flashed the probe transient occasionally on the relevant margin of the obstacle so we could quantitatively compare the extent, to which the future path and the obstacle itself are attended. The ERPs in response to visual probes on the margin of the obstacle were then compared directly to those ERPs that were evoked by visual probes on the future hand path or on the opposite, task-irrelevant side (as baseline). In this second experiment, it indeed turned out that both the future hand trajectory and the margin of the object are attended to a quantitatively similar degree during movement preparation. This favors the view that both of these locations are critical and both are visually prepared to specify the relevant movement parameters. It is difficult to differentiate whether both the obstacle and the trajectory are selected completely independently. Notably, it could also be the case that all relevant locations on the planned side are selected altogether within the same selection process.
The observed attentional selection of positions on the movement path well before movement onset suggests that obstacle avoidance implies not only the selection of the final destination and the obstacle itself. Our results, therefore, do not support the idea of obstacle avoidance being simply implemented by potential fields, which steer the effector by a combination of attracting and repulsing forces (see, e.g., Kim et al., 2017; Zhou & Li, 2014), with the optimal effector path being a dynamic combination of the repulsion from the obstacle and the attraction to the goal location (Zhou & Li, 2014). Rather, our results support the idea that either the complete intended movement path is part of a sensorimotor transformation or at least certain crucial positions along the planned movement part, which could serve as intermediate goals on the way to the final destination. Such sequencing of movement primitives has been described before in both grasping and reaching movements with multiple instructed movement components (instructed movement sequences to several goal locations).
Our paradigm could successfully probe the top–down deployment of visual attention in the scene by quantifying the strength of the neural response in visual cortex elicited by task-irrelevant visual transients (“dot probes”). Naturally this opens up the question by what type of modulating signals these observed attentional effects are brought about. What are the neural sources of the implied top–down signals? Although our current EEG study cannot answer this network question directly, there are strong arguments for neural populations in the frontoparietal network being the top–down modulators of our observed attentional weighting in visual cortex. In neuropsychological patients, for example, Milner and colleagues reported problems with obstacle avoidance after parietal damage (e.g., Milner & McIntosh, 2004; Schindler et al., 2004): In one study, patients with bilateral dorsal stream were instructed to reach between two cylinders, the exact location of which varied from trial to trial. It turned out that these patients with parietal damage could not adjust their reach trajectories according to the changing obstacle configuration, suggesting that the dorsal stream normally helps with this automatic guidance of movements around potential obstacles (Schindler et al., 2004).
Also from a human brain imaging perspective, dorsal parietal cortex seems strongly involved in the formation of obstacle avoidance trajectories. Chapman et al. (2007) reported enhanced fMRI activity of reach-related neural networks in parietal cortex when obstacle objects were present during reach movements. In a more recent study by a different group of authors (Chapman, Gallivan, Culham, & Goodale, 2011), the authors used an event-related fMRI paradigm and showed that—before movement execution—an area in the left posterior intraparietal sulcus and another area in early visual cortex were modulated by the degree of obstacle interference, suggesting that the increased activity in the intraparietal sulcus with obstacle interference provides the top–down signal to corresponding early visual areas. In a study by Lindner and colleagues, human participants were instructed to remember a set of both movement goals and positions that had to be actively avoided. Activation patterns in the posterior parietal cortex were critically involved in encoding both types of locations: those to be acquired as well as those to be avoided (Lindner, Iyer, Kagan, & Andersen, 2010).
The involvement of posterior parietal cortex in the formation of curved trajectories around obstacles has also been studied in single-cell recordings in behaving nonhuman primates (Torres & Zipser, 2002; Hoff & Arbib, 1993). Torres and colleagues, for example, recorded from single cells in posterior parietal while the monkey performed an obstacle avoidance task for which it had not been explicitly trained (Torres & Andersen, 2006). Their findings suggest the neural networks in posterior parietal are not only crucial for specifying “abstract” movement goals (i.e., the end destination of a planned reach) but also for expressing these plans as corresponding trajectories, even in joint coordinates. In a process called trajectory formation, posterior parietal cortex helps spatially and temporally organizing a sequence of hand positions required to reach the goal (Torres & Zipser, 2002). Therefore, the posterior parietal cortex also seems to encode so-called approach avoidance arm motions at multiple levels of representation (see also Torres, Quiroga, Cui, & Buneo, 2013). Also Mulliken and colleagues (Mulliken, Musallam, & Andersen, 2008; see also Hauschild, Mulliken, Fineman, Loeb, & Andersen, 2012) could successfully decode continuous trajectories from posterior parietal cortex. Such representations of the planned trajectory around an obstacle—including crucial strategic landmarks along the way—could serve well as a potential top–down source of the biasing signals we observed in visual cortex (see also Chapman et al., 2011).
This study presents some new evidence about the role of attentional landscapes (Baldauf & Deubel, 2010) in the preparation process of visual-guided reaches. In a dot probe paradigm, we could show in the signature of human EEG responses that visual attention not only selects movement goals well in advance of movement initialization but also selects locations along the intended path around an obstacle. These results indicate that the formation of attentional landscapes in the workspace is more complex than previously thought and points at possible future directions of research on their possible temporal dynamics.
We would like to thank Heiner Deubel for his support and discussions at an early state of the project.
Reprint requests should be sent to Daniel Baldauf, Center for Mind & Brain Sciences (CIMeC), Universita degli Studi di Trento Centro Interdipartimentale Mente/Cervello, Via delle Regole, 100, Rovereto, Italy, 38068, or via e-mail: firstname.lastname@example.org.