Abstract

People are typically slower when executing two tasks than when only performing a single task. These dual-task costs are initially robust but are reduced with practice. Dux et al. (2009) explored the neural basis of dual-task costs and learning using fMRI. Inferior frontal junction (IFJ) showed a larger hemodynamic response on dual-task trials compared with single-task trial early in learning. As dual-task costs were eliminated, dual-task hemodynamics in IFJ reduced to single-task levels. Dux and colleagues concluded that the reduction of dual-task costs is accomplished through increased efficiency of information processing in IFJ. We present a dynamic field theory of response selection that addresses two questions regarding these results. First, what mechanism leads to the reduction of dual-task costs and associated changes in hemodynamics? We show that a simple Hebbian learning mechanism is able to capture the quantitative details of learning at both the behavioral and neural levels. Second, is efficiency isolated to cognitive control areas such as IFJ, or is it also evident in sensory motor areas? To investigate this, we restrict Hebbian learning to different parts of the neural model. None of the restricted learning models showed the same reductions in dual-task costs as the unrestricted learning model, suggesting that efficiency is distributed across cognitive control and sensory motor processing systems.

INTRODUCTION

A central function of cognitive control is to coordinate ongoing operations associated with distinct tasks. Evidence for these control processes can be observed in the form of dual-task costs, as indexed by increases in RT and errors when individuals perform two tasks simultaneously. These costs are reduced with practice, and the magnitude of the reduction typically outpaces the reductions in single-task RTs. In fact, in some instances, the costs are no longer significant after five to eight 1-hr sessions (Hazeltine, Ruthruff, & Remington, 2006; Schumacher et al., 2001; Van Selst, Ruthruff, & Johnston, 1999). How does practicing two tasks together alter neural systems to reduce costs?

Dux and colleagues (2009) addressed this question by asking participants to practice a visual–manual (VM) task (face stimuli paired with button-press responses) and an auditory–vocal (AV) task (sounds paired with vocal responses) for 2 weeks. Participants were scanned with fMRI as dual-task costs were reduced. Participants performed trials in which one task was presented and trials in which both tasks were presented simultaneously. Figure 1 shows the RTs over 2 weeks of practice. Early in practice, participants were much slower on dual-task trials; however, by the second week, the difference between single- and dual-task trials shows a tenfold reduction from the first session. Critically, participants could perform both tasks together as quickly and accurately as they could perform either task alone (see also, Tombu & Jolicoeur, 2004; Hazeltine, Teague, & Ivry, 2002; Schumacher et al., 2001; Van Selst et al., 1999). Also of note, the amount of learning was much larger for the AV task compared with the VM task. The RTs for AV and VM tasks were nearly equal at the beginning of training but much smaller for the AV task by the end of training (see Hazeltine et al., 2006; Schumacher et al., 2001).

Figure 1. 

Data reproduced from Dux et al. (2009). (Top) The behavioral RTs over the eight sessions of the study. (Bottom) The hemodynamic response reported from IFJ, VocMC, and AC.

Figure 1. 

Data reproduced from Dux et al. (2009). (Top) The behavioral RTs over the eight sessions of the study. (Bottom) The hemodynamic response reported from IFJ, VocMC, and AC.

In the fMRI data, inferior frontal junction (IFJ) showed a stronger hemodynamic response on dual-task trials compared with single-task trials. Moreover, this difference in activation diminished over practice, closely following the pattern of dual-task costs. The fMRI data were used to evaluate three classes of models regarding the neural mechanisms supporting the transition from large initial dual-task costs to efficient multitasking. First, training might lead to a shift away from slow, deliberate processing in pFC to fast efficient processing in task-specific pathways (Kelly & Garavan, 2005; Petersen, van Mier, Fiez, & Raichle, 1998). There was, however, no increase in activation in sensory motor areas over learning to support this account. Second, training might functionally segregate the neural assemblies in pFC that are coupled to sensory motor areas, thereby creating separate processing streams for the two tasks (Erickson et al., 2007). But multivariate pattern analyses did not reveal any learning-related changes in neural activation patterns associated with different tasks as predicted by this hypothesis. On the contrary, pattern classification on IFJ revealed a decrease in task decoding over learning.

Such a decrease in task decoding is most consistent with a third hypothesis, that training improves the efficiency of information processing in pFC (Jonides, 2004). Use of higher temporal resolution fMRI data supported this account. Early in training, the peak amplitude of the BOLD response in dual-task conditions occurred approximately 500 msec later than in single-task conditions. In contrast, no differences were observed in the timing and duration of the BOLD signal at the end of training. Note that Dux and colleagues acknowledged that improved efficiency does not necessarily lie solely in pFC but could occur throughout sensory motor areas.

Given these findings, Dux, Tombu, Harrison, Rogers, Tong, & Marois (2009) concluded that response selection includes the funneling of information from sensory motor areas onto overlapping neural ensembles in pFC. These overlapping neural ensembles create a bottleneck at a central stage of response selection, which produces dual-task costs (Pashler, 1994). Training speeds up information processing in pFC, which reduces temporal processing. This proposal is consistent with neurophysiological data from macaque pFC (Asaad, Rainer, & Miller, 2000), showing a decrease in neural latency within pFC over practice. Thus, in contrast to alternative accounts of dual-task costs, practice does not diminish the role of pFC or engage alternative pathways of stimulus–response translation. Instead, practice tunes task-related processes so that they require less activation and therefore interfere less with other ongoing processes.

Dux et al. (2009) provide a detailed picture of the neural and behavioral dynamics that underlie dual-task performance and changes over learning; however, two central questions remain. First, what neural mechanism leads to the reduction in dual-task costs, that is, what mechanism causes IFJ activation to weaken and become compressed in time? Second, are changes in efficiency isolated to pFC or do they extend to sensory motor systems as well? We investigate these questions using a model-based approach to fMRI. Our goal is to simulate both RT and hemodynamic responses with the same neural process model, thereby capitalizing on the rich constraints that exist by simultaneously capturing data at two levels of analysis (White & Poldrack, 2013; Davis, Love, & Preston, 2012; Ashby & Waldschmidt, 2008; Deco, Rolls, & Horwitz, 2004). We can then use the model to test whether efficiency is isolated to cognitive control systems or is pervasive throughout the simulated neural system. We do this by turning learning off in different parts of the model and asking whether the resultant changes in the model's performance are consistent with both the behavioral and neuroimaging data. Thus, the model provides a way to bridge between brain and behavior, yielding a computationally explicit description of what IFJ does, how neural processing in IFJ changes with practice, and how these changes impact RTs and neural activation.

Current computational models of response selection would seem to provide a natural starting point for a formal account of the neurocognitive dynamics reported by Dux et al. (2009). Nevertheless, prominent models of response selection such as ACT-R (Anderson et al., 2004) and EPIC (Meyer & Kieras, 1997) are production models that operate on abstract symbols. Consequently, such models are not well situated to address questions about the neural mechanisms of learning in dual-task contexts (note that ACT-R has been used to simulate hemodynamic responses from fMRI; we discuss this approach further in the Discussion). We turn, instead, to a dynamic field theory (DFT) of response selection and motor planning proposed by Erlhagen and Schöner (2002). These researchers used dynamic neural fields (DNFs) to capture RTs across diverse response selection tasks. Critically, the theory has also been tested at the neural level using multiunit recording and ERPs (McDowell, Jeka, Schoner, & Hatfield, 2002; Erlhange, Bastian, Jancke, Riehle, & Schoner, 1999; Jancke et al., 1999).

We build on this prior work, extending the dynamic field approach to response selection using a DNF architecture that binds visual or auditory features to spatial locations and verbal labels (i.e., words; see Samuelson, Smith, Perry, & Spencer, 2011; Faubel & Schöner, 2008). We add a cognitive control system that takes its inspiration from recent work examining behavioral organization in autonomous robots (Sandamirskaya & Schöner, 2010) and executive function in young children (Buss & Spencer, in press). Moreover, we propose a linking hypothesis that maps neural activity in the model to the BOLD response. This enables us to simulate hemodynamic responses from real-time neural activity in the model. After fixing the parameters of the model to fit the behavioral data, we show how the model with the same parameters quantitatively captures neural data from Dux et al. (2009). Finally, we test whether efficiency is isolated to the cognitive control system or pervades the simulated neural system, giving us insight into the neural mechanisms that underlie efficiency over learning.

Model Architecture

The first issue to consider regarding a model of response selection and dual-task performance is how a “task” is represented. Consider a simple VM task: How do people perceive a red item on the computer screen and know to press the right response button? And how do they do this even when, for instance, the red item is presented at a left location? To do this, multiple transformations must be handled. Object locations and features are initially encoded in a retinocentric frame by neurons in the dorsal and ventral visual pathways (Mishkin, Ungerleider, & Macko, 1983). These neural representations must then be transformed from the frame of the sensors to a more abstract frame like an allocentric representation (a “central code”; Hazeltine, 2005). In addition, allocentric representations must be mapped to the frame of the motor system. Recent work describes a DNF approach to the mapping from sensors to a body-centered or allocentric spatial frame (Lipinski, Schneegans, Sandamirskaya, Spencer, & Schöner, 2012), including how object features and spatial positions are “bound” together (Spencer, Schneegans, & Schoner, in press). Moreover, the transformation from an allocentric to a motor frame has been handled within the framework of DFT (see Faubel, Dineva, & Bicho, in press). We build on this work here but simplify matters by focusing on “bound” neural representations in the spatial frame of the manual response (which we will refer to as “manual space” below to indicate, for instance, a “left” and “right” dimension corresponding to left and right button presses).

The starting point for our model of response selection comes from initial work by Johnson, Spencer, and Schöner (2008) using 2-D DNFs. Figure 2 shows a 2-D DNF that maps a stimulus dimension—color—to a response dimension—the left–right dimension of the manual response. The top panel in Figure 2A shows the excitatory layer of neurons in the response selection field; the bottom panel shows the inhibitory layer of neurons. Each 2-D layer is composed of bimodal neurons with receptive fields for both color and the location of the manual response. The vertical axis shows neurons arranged according to receptive fields for color (e.g., hue). The horizontal dimension in each panel shows neurons arranged according to their manual spatial receptive fields with neurons that “prefer” leftward responses on the left. The activation level within the excitatory and inhibitory layers is given by the inset color scale with “hotter” colors showing greater activation. Importantly, units colored red mean that they are activated above threshold. When stimulated to above-threshold levels, the excitatory layer passes activation to the inhibitory layer. Similarly, when the inhibitory layer is activated to above-threshold levels, broad inhibition is passed back into the excitatory layer. Together, these layers implement a form of local excitation/surround inhibition common in neural systems (Amari, 1977).

Figure 2. 

An example of a 2-D system highlighting the fast and flexible association created by directly binding information along different dimensions and the Hebbian learning process operating within DNFs. The excitatory response selection field is shown on top and the inhibitory field is shown on the bottom. Excitatory connections are shown with green arrows and inhibitory connections are shown with red arrows. (A) The fields are at resting level (the mottling is because of spatially correlated noise). (B) The fields just after inputs corresponding to a specific color and spatial are turned on. This reflects the onset of a colored item at a rightward location. (C) The model after a peak has been created in the response selection field. (D) The fields after the inputs have been turned off, and the fields have returned to resting level. The influence of Hebbian learning can now be seen by the localized boost at the location of the peak. (E) The model is given only a color input. (F) The model has formed a peak of activation binding the color to the associated spatial location.

Figure 2. 

An example of a 2-D system highlighting the fast and flexible association created by directly binding information along different dimensions and the Hebbian learning process operating within DNFs. The excitatory response selection field is shown on top and the inhibitory field is shown on the bottom. Excitatory connections are shown with green arrows and inhibitory connections are shown with red arrows. (A) The fields are at resting level (the mottling is because of spatially correlated noise). (B) The fields just after inputs corresponding to a specific color and spatial are turned on. This reflects the onset of a colored item at a rightward location. (C) The model after a peak has been created in the response selection field. (D) The fields after the inputs have been turned off, and the fields have returned to resting level. The influence of Hebbian learning can now be seen by the localized boost at the location of the peak. (E) The model is given only a color input. (F) The model has formed a peak of activation binding the color to the associated spatial location.

The model forms a real-time neural representation by building a stable “peak” of activation centered over particular sites in the field (see hot spot of activation in Figure 2C). This is achieved through the balance of excitation and inhibition within the field. For example, in Figure 2B, the model is given two inputs to the excitatory layer. One is a ridge running horizontally specifying a color value (e.g., red). The other is a ridge running vertically specifying a manual response (e.g., push the right button). We assume that these ridges reflect input from cortical fields earlier in the visual pathways that detect the object's color (for details, see Spencer et al., in press) and task instructions that specify the nature of the manual response. In Figure 2C, activation has pierced threshold, and neural interactions have been engaged to build a peak of activation for neural units tuned to the particular stimulus–response information presented in the inputs—the model has mapped the red color to a rightward response.

Critically, the DNF model has a Hebbian learning process to capture changes in neural dynamics over learning. For example, in the response selection field, this process accumulates memory traces while a peak of activation is present, and decays memory traces over a slower timescale when no peak is present. To illustrate this, Figure 2D shows the DNF model after the inputs have been turned off and the layers have settled back to the neural resting level. In the top panel, a memory trace is visible at neural sites corresponding to where the peak was present in Figure 2C, boosting the resting level of these neural units. An important consequence of this is shown in Figure 2EF. Here, the model is only given the color information that was present on the previous trial (i.e., there is only a horizontal ridge corresponding to red). Nevertheless, in Figure 2F, the model recalls that the red object requires a rightward response—even after only a single learning trial. This occurs because the Hebbian trace brings the “red-right” neural units closer to the activation threshold. Thus, when the red color is presented, the “red-right” units reach threshold first and win the competition to build a peak. Although the model builds a memory trace after a single trial, the further accumulation of Hebbian traces over practice produces a decrease in peak latency (i.e., a decreases in RT) as activation at particular stimulus–response combinations is brought closer to threshold.

The 2-D field in Figure 2 can quickly learn stimulus–response mappings. Critically, this model can generalize to multiple feature dimensions (e.g., visual, auditory, tactile) using the space of the manual response as a binding dimension (Spencer et al., in press; Johnson et al., 2008). This expanded architecture is shown in the left panels of Figure 3, which show two 2-D response selection fields—a shape-manual space field (Figure 3A) and a color-manual space field (Figure 3B). These fields pass activation back and forth along the shared manual space dimension. The peak on the right side of the field in Figure 3A—which maps, for instance, a star shape to a rightward response—passes input into the right side of the field in Figure 3B—which maps a red color to a rightward response. Thus, the pattern of peaks in Figure 3A and B capture the “bound” representation that the red star goes on the right (see, e.g., Buss & Spencer, in press).

Figure 3. 

Architecture of word-binding model from Samuelson et al. (2011). (A and B) Two dynamic neural fields that are tuned to a shape or color dimension but share a manual space dimension. (C and D) Two dynamic neural fields that are also tuned to shape or color but share a vocal label dimension.

Figure 3. 

Architecture of word-binding model from Samuelson et al. (2011). (A and B) Two dynamic neural fields that are tuned to a shape or color dimension but share a manual space dimension. (C and D) Two dynamic neural fields that are also tuned to shape or color but share a vocal label dimension.

The left panels in Figure 3 show a model that can build fast associations between multiple stimulus dimensions and a manual spatial response. Critically, however, stimulus–response associations can extend beyond manual responses into the vocal modality. Panels C and D of Figure 3 show a second part of the DNF architecture taken from work by Samuelson et al. (2011) and Faubel and Schöner (2008). This part of the neural architecture consists of two 2-D neural fields, whose peaks are bound to a second binding dimension—vocal labels or words. Sites within each 2-D field have bimodal receptive fields. They are receptive to input from label neurons—population representations of a spoken word—and featural input from, for instance, early visual areas. A Hebbian process enables the model to learn feature–label associations quickly and influence performance on subsequent trials. Finally, because the fields in Figure 3A and C share a common feature dimension—the shape dimension—they pass activation back and forth along this dimension. Similarly, the fields in Figure 3B and D share a common feature dimension—color—and pass activation back and forth along this dimension. This is reflected in the horizontal ridges of activation in the shape-label field and the color-label field. In this way, the model can recall the vocal labels associated with presented object features. For instance, in Figure 3C, the model is recalling that the object is a “star,” whereas in Figure 3D, it is recalling that the object is “red.” Note that neural interactions across the label dimension are winner-takes-all with sharp boundaries between one label and the next (i.e., local excitation spreads minimally from one unit to the next). Such interactions thus capture the discrete nature of word representations common in connectionist models of early word learning (Damasio, 1989; Gross, Rocha-Miranda, & Bender, 1972).

To extend the DNF model used by Samuelson et al. (2011) to a dual-task setting, we need to address how to selectively attend to particular stimulus–response dimensions, for instance, mapping a color dimension to a manual response and a tone dimension to a vocal response. Figure 4 shows a generalized version of the response selection model with four response selection fields—a VM field, an auditory–manual field, a visual–vocal field, and an AV field—along with a type of cognitive control system we will refer to as the dimensional attention network.

Figure 4. 

Architecture of dual-task model. Green lines show excitatory connections, and red lines show inhibitory connections. The black lines denote the gating function of the CoS neurons. For the simulations of the data reported by Dux and colleagues, only the VM and AV components are used.

Figure 4. 

Architecture of dual-task model. Green lines show excitatory connections, and red lines show inhibitory connections. The black lines denote the gating function of the CoS neurons. For the simulations of the data reported by Dux and colleagues, only the VM and AV components are used.

The dimensional attention network depicted in the center of Figure 4 consists of dimensional attention units inspired by recent work on executive function and dimensional attention with children (Buss & Spencer, in press). These units boost baseline levels of activation within the response selection fields, enabling the selection of task goals (respond to the visual stimulus with a manual response) and flexibly switching between alternative goals (switch to a vocal response based on an auditory stimulus). For instance, when an input is presented to the VM field in the top left of Figure 4, this field passes activation to its associated attention unit (Att). As this unit is activated, it projects a global boost of activation back onto the VM field. This facilitates peak formation in the VM field, enabling the system to selectively attend to particular set of stimulus–response bindings. The attention units are self-excitatory (see green arrows in Figure 4) and mutually inhibitory (see red arrows in Figure 4) consistent with lateral inhibitory interactions in frontal cortex (Fuster, 1989). This excitatory and inhibitory coupling leads to winner-takes-all interactions among these units—once a unit achieves above-threshold activation and goes into the “on” state (analogous to a peak in a neural field; see Schoner & Schutte, in press), the other units are suppressed.

The attention units in Figure 4 bias the system to selectively build responses based on particular response selection goals, but to respond to multiple stimulus–response bindings, a mechanism is also needed to decide when one goal has been achieved and another goal can be attended. Here, we borrow a concept from autonomous robotics called the “condition of satisfaction” (CoS; Sandamirskaya & Schöner, 2010; Searle, 1980). Conditions of satisfaction are the perceptual or motor states that indicate when a goal has been met. For instance, if the goal is to push a button when a specific color is presented, then the neural system must monitor motor output to sense when the finger has done its job. For simplicity, we assume that building a peak within one of the response selection fields in Figure 4 is sufficient to trigger the offset of a goal (see Bausenhart, Rolke, Hackley, & Ulrich, 2006). To detect the presence of a peak in a response selection field, we add CoS units to the model. Each CoS unit is activated by peaks within an associated 2-D field. When a CoS unit is activated to above-threshold levels, it gates input from the 2-D field to the associated attention unit. This prevents input to the attention unit, allowing the unit to return to the neural resting level, and releases the current focus of attention. Once the active attention unit goes into the resting state, the other attention units are released from inhibition and another attention unit can become activated. This is consistent with data showing that satisfying the conditions of one task enables the processing of the other task early in practice (Bausenhart et al., 2006).

The rise of activation of the CoS units are slower than the attention units (see Appendix), so the attention units become engaged when activation is initially building in the response selection fields, whereas the CoS units only become engaged once this activation has robustly gone above threshold. Hebbian learning processes are engaged for the attention and CoS units. Similar to Hebbian learning within the response selection fields, Hebbian learning within these units serves to boost their resting level so that they are closer to threshold as they are repeatedly activated within a task.

In summary, the model initiates response selection by binding stimuli and responses within 2-D fields tuned to specific combinations of stimulus dimensions (e.g., visual or auditory) and response dimensions (e.g., manual space, vocal labels). Learning is accomplished via a fast and flexible Hebbian mechanism operating within the response selection fields, which serves to drive the formation of stimulus–response associations and to strengthen these associations from trial to trial. The response selection fields are modulated by an attentional network. Attention units receive activation from the response selection fields when a stimulus is presented. Once activated, the attention units boost the resting level of the associated response selection field, facilitating the formation of a peak and leading to the generation of a response. Finally, CoS neurons release attention from a task goal once the response has been activated. Together, the attention and CoS units enable the system to selectively attend to one task goal, achieve that goal (i.e., form a peak), and then attend to any remaining goals.

Dynamics of Response Selection

In this section, we show simulations of the DNF model of response selection, highlighting how the model captures the neural processes that underlie dual-task performance. Recall that Dux et al. (2009) paired a VM task (pressing buttons in response to different images of faces) with an AV task (vocalizing a response to different tones). Thus, for the simulations below, we only use the two response selection fields (and associated attention and CoS units) needed to accomplish these two tasks.

The top panel of Figure 5 shows the activation of the attention and CoS units during the course of a dual-task trial early in training. Panels A–E show activation in the response selection fields at key points during the trial (see time labels in milliseconds) to highlight the model's neural dynamics. The horizontal axis in the response selection fields shows the response dimension (manual or vocal responses), whereas the vertical axis shows the stimulus dimension (visual or auditory features). The activation level is given by the color inset.

Figure 5. 

The dual-task model in action on a dual-task trial. Top: The activation of the attention and CoS neuron over the course of the trial. (A) The VM and AV response selection fields before the onset of the stimulus. Here, the stimulus–response mappings are shown for each task. (B) The presentation of a stimulus to each response selection field. At the corresponding point in the time course of activation of the attentional network, the attentional neurons for each task has become robustly activated. At this point, they are competing with one another through their mutual inhibitory coupling. (C) A peak has formed in the VM response selection field. At the corresponding time point, the VM-CoS neuron becomes activated and begins to turn off the interaction between the AV-attention neuron and AV response selection field. (D) A peak has been built corresponding to the selection of a response within the AV response selection field. At the corresponding time point, the AV-CoS neuron becomes activated and begins to turn off the interaction between the AV-attention neuron and AV response selection field. (E) The response selection systems have returned to resting level, and the influence of Hebbian learning can be seen by the increased activation of the stimulus–response mappings for the previously activated SR pairs.

Figure 5. 

The dual-task model in action on a dual-task trial. Top: The activation of the attention and CoS neuron over the course of the trial. (A) The VM and AV response selection fields before the onset of the stimulus. Here, the stimulus–response mappings are shown for each task. (B) The presentation of a stimulus to each response selection field. At the corresponding point in the time course of activation of the attentional network, the attentional neurons for each task has become robustly activated. At this point, they are competing with one another through their mutual inhibitory coupling. (C) A peak has formed in the VM response selection field. At the corresponding time point, the VM-CoS neuron becomes activated and begins to turn off the interaction between the AV-attention neuron and AV response selection field. (D) A peak has been built corresponding to the selection of a response within the AV response selection field. At the corresponding time point, the AV-CoS neuron becomes activated and begins to turn off the interaction between the AV-attention neuron and AV response selection field. (E) The response selection systems have returned to resting level, and the influence of Hebbian learning can be seen by the increased activation of the stimulus–response mappings for the previously activated SR pairs.

Figure 5A shows the instructed stimulus–response mappings within VM and AV fields. These are subthreshold “bumps” of activation that are priming particular responses for the relevant stimuli of the task (e.g., faces matched to two different button presses and sounds mapped to vocal responses). At the start of the trial (after 80 msec; note the rise in activation of the Att nodes at this time point), a stimulus is presented as a ridge of activation at a particular stimulus location extending across all possible responses. Figure 5B shows ridges of activation forming 480 msec after a face and a sound are presented but before responses are selected. Because both tasks were simultaneously presented, the attention units both rise toward threshold (0 activation—see dashed line in the top panel), but the increase in activation is slow because of their mutual inhibitory coupling. In Figure 5C, the VM field has formed a peak of activation, selecting a label for the face stimulus. At this point in the top panel, the VM-CoS unit becomes activated, turning off input to the VM-attention unit and driving this attention unit below threshold. This frees the system to robustly engage the AV task, which is reflected by the boost in activation of the AV-attention unit around 700 msec. That is, the model is able to fully engage the AV task without inhibitory competition from the VM task. In Figure 5D, the model has now formed a peak within the AV field. At this point in time, the AV-CoS unit becomes activated and turns off input to the AV-attention unit. Finally, in Figure 5E the model has returned to its resting state, and the influence of Hebbian learning from the previous trials can be seen (note the brighter color of the just-selected stimulus–response associations in Figure 5E relative to Figure 5A). Note that the same sequence of events play out for single-task trials, except there is no competition in activation for the attention units because they are activated individually based on the task for a given block.

SIMULATING DUX ET AL. (2009)

The model provides quantitative simulations of the behavioral and hemodynamic data presented by Dux et al. (2009). We first describe the simulation of the behavioral task. After setting the model parameters to fit the behavioral data, we adapt a method for generating hemodynamic responses from integrate-and-fire networks (see Deco et al., 2004) to the DNF model. This allows us to simulate BOLD responses from real-time neural activity of the model. Next, we evaluate whether different components of the model effectively capture hemodynamic data from Dux et al. Finally, we manipulate where learning takes place in the DNF model to determine the loci of learning that are critical for driving the behavioral practice effects.

Methods

Simulations were conducted in Matlab 7.5.0 (Mathworks, Inc., Natick, MA) on a PC with an Intel i7 3.33 GHz quad-core processor (the Matlab code is available from the authors on request). The model was given the same training regimen as described by Dux et al. (2009). This includes eight sessions in which dual-task and single-task trials for VM and AV tasks were randomly intermixed. Each session consisted of 150 trials. This is fewer trials than participants were given in the study by Dux et al. (2009). Given the time-intensive nature of the simulations, we chose to simplify the sessions by including fewer trials. This is reasonable given the nature of the Hebbian learning mechanism we are using, essentially, the number of trials and the strength of learning scale linearly. To reduce the number of trials then, we increased the learning rate (by decreasing the tau parameter; see τw_build, τd_build, τΩ_build, τw_decay, τd_decay, τΩ_decay parameters in Appendix).

To begin the simulation work, the excitatory and inhibitory parameters for the fields and attention/CoS nodes were tuned to fit the initial session of single-task data from Dux et al. These parameters were then tested in dual- and single-task conditions over learning. Data were averaged over a batch 10 simulations (corresponding to 10 individual participants). A total of 12,000 trials were simulated. RTs from the model were determined by the number of time steps from the onset of a stimulus until a peak of activation formed within the excitatory layer of a response selection field (i.e., activation > 0). For the purpose of mapping time steps in the model to real time, 1 time step was set equal to 1.6 msec. In previous work (Spencer, Barich, Goldberg, & Perone, 2012), this mapping was set to 1 time step equals 0.56 msec. We selected a mapping here to ensure that we sample time densely enough to accurately capture RTs. Critically, the same value was used to map RTs and the hemodynamic response from the model. As noted previously, there are inherent differences in RTs associated with VM and AV tasks (Hazeltine et al., 2006; Schumacher et al., 2001). To capture this, we modified the relaxation time parameter of the VM field to be four times slower than the AV field (reflected by the tau parameters in the model equations; see τe and τi parameter in Appendix). This might reflect observed differences in speed of processing with auditory versus visual cortex (Cohen, Horowitz, & Wolfe, 2009). To compensate for this timescale difference, the resting level of the excitatory layer was 9% closer to threshold and the stimuli were 12.5% stronger for the VM field compared with the AV field. All other parameters—including all parameters that affect learning over trials—were the same (see Table 3).

Results

As shown in Figure 6, the model reproduces three important features of the behavioral data: initial dual-task costs, reduction of these costs over practice, and differential learning rates for the different modality pairings. The dual-task costs emerge from the shared inhibition among the attention nodes. This led to a slower rise in activation of the attention units on dual-task trials, producing a slower peak-build time in the response selection fields. The reduction of these costs over practice results from Hebbian learning operating both within the response selection fields as well as within the attention system.

Figure 6. 

Behavioral fits of the model and the data from Dux et al. (2009).

Figure 6. 

Behavioral fits of the model and the data from Dux et al. (2009).

Figure 7 shows the activation of the attention units during single and dual-task trials during Session 1 and Session 8. As can be seen, Hebbian learning serves to compress the competition of the attention units in time. That is, early in practice, the VM task is completed first, with a substantial lag before the AV task is initiated. Late in practice, by contrast, the activation peak in the AV-attention unit is quickly followed by a rise in activation of the VM-attention unit. In this situation, dual-task costs are greatly reduced. Note that the compression in time on dual-task trials is also accompanied by a switch in the task ordering similar to what is seen in the behavioral data. That is, early in practice, the VM task is completed before the AV task on dual-task trials. By Session 3, however, the AV task is completed more quickly than the VM task. The early advantage for the VM task stems from the higher resting level and stronger inputs. As Hebbian learning accumulates, however, the faster timescale of the AV task allows the AV-attention unit to outpace the VM-attention unit. This leads to an emergent switch in the task ordering on dual-task trials.

Figure 7. 

Attention unit activation on single-task (solid lines) and dual-task (dashed lines) trials during Session 1 (top) and Session 8 (bottom).

Figure 7. 

Attention unit activation on single-task (solid lines) and dual-task (dashed lines) trials during Session 1 (top) and Session 8 (bottom).

In this way, the model captures an account of dual-task performance, mostly consistent with the view offered by Dux et al. (2009), that costs are eliminated as both response selection processes and control processes become more efficient through Hebbian learning. This qualitative change in performance occurs without a shift in the underlying response selection mechanisms; no components were eliminated, and no task operations ceased to interfere with other operations (i.e., became automatic). The reduced dual-task costs are consistent with a latent bottleneck account (e.g., Ruthruff, Johnston, Van Selst, Whitsell, & Remington, 2003), in which some task operations are performed serially but, because they occur at different times, there is no observable increase in RT.

The model also captures the different learning rates for VM and AV tasks. These differences in performance are not a direct result of differences in the strength of Hebbian memories. As shown in Figure 8, there is a stronger build-up of memory traces in the VM response selection system and on the VM-attention and CoS units. Nevertheless, VM RTs change less over practice (see Figure 6), because these stronger memory traces are embedded within relatively slow neural dynamics (see τe and τi parameters in Appendix). Thus, learning is not just a function of memory strength but also of the neural dynamics within which these memories are situated. Interestingly, the slower timescale of the VM response selection system makes peaks of activation persist for a longer duration late in practice when Hebbian traces are strong (see Figure 7). We return to this point in the Discussion.

Figure 8. 

Accumulation of Hebbian memories on different components of the model over the eight sessions of training.

Figure 8. 

Accumulation of Hebbian memories on different components of the model over the eight sessions of training.

FROM REAL-TIME SIMULATIONS OF NEURAL ACTIVITY TO SIMULATED HEMODYNAMICS

A primary goal of this work is to simulate both behavior and the associated hemodynamic responses over training. Thus, we now consider how the dynamics of the model can be mapped onto the target neural measure—the BOLD response measured with fMRI. Simulating the real-time dynamics of both neural and behavioral dynamics has been a challenging endeavor (Ashby & Waldschmidt, 2008). To achieve this using DFT, we adapt an approach based on biophysical work exploring the neural basis of the BOLD signal. Logothetis, Pauls, Augath, Trinath, and Oeltermann (2001) recorded single- and multiunit data along with local field potentials (LFP) and the BOLD signal in visual cortex of macaques while they were shown checkerboard displays. An LFP is a measure of dendritic activity over a localized population of neurons, accounting for changes in both inhibitory and excitatory ion channels. This provides a measure of the input to and local processing within a region of cortex. Logothetis et al. (2001) reported that the LFP was most strongly correlated with the BOLD response. Furthermore, by convolving the LFP with an impulse response function (specifying the time course of the slow blood flow response to neural activity), the authors were able to reproduce the BOLD signal. This suggests that the LFP is a strong contributor to the neural signal driving the BOLD response.

Deco et al. (2004) expanded upon this work using an integrate-and-fire network to simulate the hemodynamics underlying visual working memory. They estimated an LFP by summing the absolute value of the total synaptic current flow into the cells, including excitatory components (implemented through NMDA and AMPA receptors) and inhibitory components (GABA). According to the model, these components are the primary contributors to the rate of change in membrane potential, excluding the stability term and factors that influence the resting potential. By convolving this estimated LFP with an impulse response function, the authors were able to simulate the BOLD response in a visual working memory task.

Here we adapt the approach of Deco et al. (2004) for use with DNFs. In particular, we created a DFT-LFP measure by summing the absolute value of all terms contributing to the rate of change in activation within a DNF (i.e., within a component of the model), excluding the stability term and the two factors that impact the neuronal resting level—a resting level parameter (h) and the Hebbian memory traces, which modulate the baseline resting levels over learning. The included terms reflect excitatory and inhibitory interactions within each component of the model, excitation between components of the model, and noise (see Appendix for equations). Note that we excluded the stimulus input in the LFP computation because we applied inputs directly to the model rather than implementing these in a more neurally realistic manner (e.g., by using simulated input fields as in Spencer et al., in press).

In the simulations below, we compare the hemodynamics of the AV field to activation in auditory and vocal-motor cortex (VocMC). Moreover, because the attention units are critically involved in the coordination of dual-task performance, we probe whether the hemodynamics generated from these units correspond to IFJ activation.

Methods

A DFT-LFP was computed for each of the components of the model at each time step: the LFP from the AV response selection system (combining the excitatory and inhibitory layers) and the LFP from the attention units (summed across both the AV and VM attention units). Note that two additional LFPs were computed—one for the VM response selection system and one for the CoS units—but these hemodynamics are not discussed in detail below for ease of comparison with the set of data reported by Dux et al. (2009).

After recording the DFT-LFP components at each time step, an intertrial interval was artificially added between each trial by inserting a vector of zero's equivalent to a 1-min duration. This yielded a robust estimate of the hemodynamic response on each trial, allowing the hemodynamic response to fully return to baseline. Each LFP time course was then convolved with a sum of Guassians function (Deco & Rolls, 2005; Deco et al., 2004; see Appendix). The resulting data were then analyzed by initializing the hemodynamic response at the start of each trial to a 0 baseline value and normalizing the simulated hemodynamics by computing the mean hemodynamic response for each session and dividing by the maximum value across sessions. This was done separately for each component of the model yielding a localized normalization similar to what is done in fMRI. The average hemodynamic response across trial types (single AV, single VM, and dual task) was then calculated for each session.

Results

Dux et al. (2009) focused their discussion on the change in activation in the IFJ with practice (reproduced in Figure 9A); thus, we begin by comparing IFJ activation to the hemodynamics generated from the model's attention system. The attention units showed a strong hemodynamic response on dual-task trials relative to single-task trials early in learning. Over practice, this difference reduced to single-task levels (Figure 9B). This bears a close correspondence to the data reported from IFJ, suggesting that the functional role of the attention units and how these units change with practice provide a comprehensive explanation for both the changes in behavior and the changes in neural activation in IFJ. That is, not only does the model account for the behavioral changes that result from dual-task practice, but it also provides a quantitative explanation for the changes in activation that accompany the improvements in performance.

Figure 9. 

Hemodynamic data from Dux et al. (2009) (A) and the hemodynamic fit from the attentional neurons of the model (B; note that the red and blue lines overlap in this panel).

Figure 9. 

Hemodynamic data from Dux et al. (2009) (A) and the hemodynamic fit from the attentional neurons of the model (B; note that the red and blue lines overlap in this panel).

The top two rows of Figure 10 show data from Dux et al. (2009) for the VocMC and the auditory cortex (AC). These cortical areas showed robust hemodynamic responses when the AV task was engaged, but no differences across single- and dual-task contexts and no changes in the hemodynamics over learning. As can be seen in the bottom row of Figure 10, the hemodynamics from the AV response selection system paralleled these patterns. Thus, the task-specific components of the model showed a similar pattern of activation to the task-specific cortical areas reported by Dux et al. (2009). Note that a similar pattern of hemodynamics was observed with the VM task fields.

Figure 10. 

(A) Data from vocal–motor cortex and auditory cortex reported by Dux et al. (2009). (B) Hemodynamic data from task-specific response selection systems from the DNF model.

Figure 10. 

(A) Data from vocal–motor cortex and auditory cortex reported by Dux et al. (2009). (B) Hemodynamic data from task-specific response selection systems from the DNF model.

The first row of Table 1 shows the RMSE value for the simulated behavioral and hemodynamic data. Columns 1 and 2 show the RMSE for the single-task RTs. Over learning, the model diverged from the behavioral data by an average of 15.15 and 18.03 msec for the single-AV and single-VM tasks, respectively. As shown in Column 3, the model diverged from the dual-task behavioral data by an average of 15.74 msec (averaged across both the AV and VM tasks over learning). Finally, Columns 4 and 5 show the RMSE of the simulated hemodynamics from IFJ and VocMC. These values were computed across the entire time course of the hemodynamic response and averaged across all trial types. On average, the model diverged from the hemodynamic data by 0.144% signal change for both the IFJ and VocMC data.

Table 1. 

RMSE for the Behavioral and Hemodynamic Fits of the DFT Model


Single-AV
Single-VM
Dual
IFJ-fMRI
VocMC-fMRI
Full Hebbian learning 15.15 18.03 15.74 0.144 0.144 
Hebbian learning on Att units 24.71* 39.02* 43.27* 0.147* 0.167* 
Hebbian learning in RS layers 150.44* 48.30* 161.31* 0.149* 0.141* 
Stronger Hebbian learning on Att units 15.65 27.67* 28.70* 0.148* 0.138* 

Single-AV
Single-VM
Dual
IFJ-fMRI
VocMC-fMRI
Full Hebbian learning 15.15 18.03 15.74 0.144 0.144 
Hebbian learning on Att units 24.71* 39.02* 43.27* 0.147* 0.167* 
Hebbian learning in RS layers 150.44* 48.30* 161.31* 0.149* 0.141* 
Stronger Hebbian learning on Att units 15.65 27.67* 28.70* 0.148* 0.138* 

Column 1 shows the RMSE for single-task RTs over learning in the AV task. Column 2 shows the RMSE across single-task RTs over learning in the VM task. Column 3 shows the RMSE across dual-task RTs over learning in both the AV and VM tasks. Column 4 shows the RMSE between the Att nodes and IFJ HRF time course over the three phases of learning in both single- and dual-task conditions. Column 5 shows the RMSE between the AV fields and VocMC HRF time course over the three phases of learning in both single- and dual-task conditions.

*p < .05 compared with the Full Hebbian learning model.

The DNF model effectively captures both behavioral and neural data from the study by Dux et al. (2009), including changes in neural efficiency over learning. This demonstrates that a Hebbian mechanism can underlie behavioral and neural changes in dual-task performance. The simulations that generated the data in Figures 6,78910 included Hebbian learning in the attentional units, the CoS units, and the response selection fields (see Figure 8). But are all of these Hebbian changes critical to increases in “efficiency” over practice?

We probed this by examining which aspects of learning were most essential for producing reductions in dual-task costs. In particular, we ran additional batches of simulations that isolated learning to either the attentional network (i.e., learning is turned off within the VM and AV fields) or to the response selection fields (i.e., learning is turned off within the attentional and CoS network). Rows 2 and 3 of Table 1 shows the RMSE for 10 iterations each of these altered models. RMSE values for these versions of the model were compared with the full Hebbian model using a t test. RMSE values that were significantly different from the full model shown in the top row are marked with an asterisk. When learning was isolated to the attention units (Row 2), the RMSE was significantly larger for all fits. These fits were poorer, however, when learning was isolated to the response selection system (see Row 3). Interestingly, the fit for the VocMC hemodynamics was significantly better in this situation. Although there are significant deviations in performance when learning is isolated to either component of the model, the pattern of results shown in Table 1 suggest that learning on the attentional network more strongly influences the AV task, but learning within the stimulus response systems more strongly influences the VM task. Specifically, learning only on the attention network produced a better RMSE for the single-AV task than the single-VM task as well as better fits of the VocMC hemodynamics. Learning only in the RS layers, however, produced a better RMSE for the single-VM task than for the single-AV task.

One challenge in interpreting the simulation results in Table 1 is that the DNF model is a fully recurrent dynamical system; thus, a change in one component of the system is likely to have a system-wide impact. Thus, in a final simulation, we examined whether we could improve the fits of the model when learning was isolated to the attention and CoS units. In particular, we resimulated the model using 10% stronger Hebbian memories on these units. Can stronger learning within the attentional network compensate for a lack of learning within the RS layers? As can be seen in the bottom row of Table 1, this manipulation significantly improved the fit of the single-AV RTs relative to the full model but still poorly fit the single-VM task and dual-task RTs as well as the IFJ hemodynamics. Thus, learning is necessary within both the attention network and the RS layers, suggesting that changes in dual-task “efficiency” over practice are distributed across both cognitive control and sensory motor processing areas.

DISCUSSION

Using basic neural principles, we were able to simulate the quantitative details of a study of dual-task performance over practice. The critical phenomena addressed by the model were behavioral and neural. Behaviorally, participants reduced both single-task and dual-task RTs with practice, and the reductions were much greater for the dual-task trials. Neurally, the IFJ showed a pattern of activation that closely followed dual-task costs. Early in training, activation on dual-task trials was significantly larger than single-task trials. By the end of training, there was no difference between single- and dual-task activation.

We used the DNF model to probe two questions regarding the increased efficiency at the behavioral and neural levels. The first issue was the mechanism underlying changes in efficiency. Although the data reported by Dux and colleagues (2009) characterized the neural dynamics accompanying learning, the processes giving rise to such changes remained ambiguous. The DNF model implemented a simple form of Hebbian learning that provides a computationally explicit account of changes in RT. This same model also generated quantitative estimates of the changes in neural activation resulting from these learning mechanisms. Thus, the DNF model demonstrates that a Hebbian process is sufficient to create changes in dual-task efficiency.

The second issue we addressed was the locus of efficiency and whether it extends beyond IFJ. We explored this by restricting learning to the attention network or to the response selection fields and comparing changes in the pattern of RTs and hemodynamics over practice. Results revealed that efficiency through Hebbian learning was important both at the level of the attentional units and the level of the response selection system, suggesting that the locus of efficiency is distributed across the cortical areas involved in these tasks. This conclusion is broadly consistent with the wealth of data demonstrating that plasticity is ubiquitous throughout the brain; learning-related changes in neural firing have been observed in nearly every region where they have been sought.

The DNF account of dual-task performance examined here shares many similarities to the explanation offered by Dux et al. (2009). The attention system resembles the overlapping neural ensembles within IFJ posited by Dux et al. (2009) with laterally inhibitory connections between the ensembles (i.e., the attention units). The implementation of these neural ensembles in a neural process model specifies the functional role they play in dual-tasking and response selection. The attention network operated by boosting activation for different task representations. A processing “bottleneck” emerged from the inhibitory competition between these attention units. The “efficiency” displayed at the end of training was brought about through Hebbian learning, which boosted the baseline activation levels, compressing activation in time, and allowing the system to execute both tasks in rapid succession.

An important component of the implementation of dual-task costs in the model is the reciprocal inhibition between attention units. In a dual-task context, these inhibitory connections are detrimental to performance and ultimately must be overcome through Hebbian processes. These inhibitory connections, however, are important for implementing task switching processes (Buss & Spencer, in press). By activating a newly relevant task goal, inhibitory interactions can suppress other previously active or competing task goals (see, e.g., Sandamirskaya & Schöner, 2010). This suggests that dual-task training, while benefiting performance in a multitasking context, may have consequences for other aspects of behavior.

Although the model presented here is the first neural process model of response selection to integrate behavioral and neural data, other models have pursued this goal using alternative means. For example, Anderson and colleagues extended the ACT-R framework—a general model of human performance—to demonstrate how symbolic systems can interface with the subsymbolic distributed processing thought to underlie brain function (Jilk, Lebiere, O'Reilly, & Anderson, 2008). Consistent with this extended framework, these researchers used ACT-R to capture and predict fMRI data in different types of tasks (Anderson, Qin, Jung, & Carter, 2007). The linking hypothesis used in this case assumed that the duration of activation of modules in ACT-R determined the magnitude of the hemodynamic response.

This work suggests an alternative way to link computational models to fMRI data. Thus, we examined whether the linking hypothesis proposed by Anderson et al. might be an effective way to map the DNF model to fMRI data from Dux et al. (2009). We computed the duration of time on each trial that the attention units showed activation > 0 over practice. Figure 11A shows how a key neural measure used in the present report—the DFT-LFP from the attention units—changed over the course of training across single and dual-task conditions. Figure 11B shows the average duration of above threshold activation for the attention units. There is a drastic departure from the pattern of change over learning reported by Dux et al. Both the AV and VM attention neurons are activated for a longer duration on single-task trials compared with dual-task trials. One might assume that dual-task trials are harder and require a longer period of activation. Instead, durations are shorter on dual-task trials because of the active inhibitory coupling between the attention units, which deactivates the units more quickly. In addition, it is clear in Figure 11B that the changes in durations over learning provide a poor match to the data reported by Dux et al.—some durations increase over learning (VM task) whereas others decrease (AV task). Thus, in the context of our neural process model, the linking hypothesis proposed by Anderson et al. does not provide an effective way to map brain and behavior.

Figure 11. 

DFT-LFP (A) and average duration of activation of attention neurons (B) for different trial types over learning.

Figure 11. 

DFT-LFP (A) and average duration of activation of attention neurons (B) for different trial types over learning.

ACT-R has been used to simulate dual-task data similar to the effects examined here (Anderson, Taatgen, & Byrne, 2005). In these simulations, Anderson and colleagues suggest that the elimination of dual-task costs results from a “latent bottleneck.” That is, the two tasks are performed with sufficient lag that they do not try to access the bottleneck at the same time (Ruthruff et al., 2003). The present model speaks to this idea. Specifically, as can be seen in Figure 7, the activation of the AV Att unit is suppressed before the VM task is completed in a single-task situation. Thus, the AV Att unit is deactivated before any strong interference can be observed for the VM task. It is an open question whether the model possesses a strict latent bottleneck; although there is no direct competition for computational resources as in a standard bottleneck model (e.g., Pashler, 1994), the attentional units are strongly competitive and, therefore, place constraints on serial processing of the tasks. This type of constraint is akin to constraints in EPIC, a general computational model of human performance (Meyer & Kieras, 1997), which holds that dual-task costs are reduced as the strategic delays are eliminated with practice. As in DFT, EPIC models reductions in dual-task costs without qualitative changes in the way responses are selected as practice unfolds. However, unlike EPIC, the DNF model specifies quantitative changes in the selection processes that lead to reduced costs.

Despite these various points of conceptual agreement that exist between DFT and current theories of response selection, there are important points of divergence as well. Most critically, DFT specifies a neutrally grounded account that clarifies the real-time processes of response selection as well as candidate neural mechanisms of learning. Moreover, we specify a direct way to link neural processes in the model to functional neuroimaging. These differences may offer ways to test competing predictions of models of response selection in future work at both the behavioral and neural levels. Our exploration of the ACT-R linking hypothesis with DFT is suggestive on this front—it is not the case that the ACT-R method of fitting fMRI data works with our model. Thus, tangible differences between models and linking hypotheses lead to differences in model outcomes that could be tested experimentally in future work.

A second broad framework that addresses how response selection is carried out is the LEABRA model developed by O'Reilly and colleagues (Hazy, Frank, & O'Reilly, 2007; O'Reilly, Noelle, Braver, & Cohen, 2002). LEABRA takes a neural computational perspective using connectionist-style modeling and addresses control processes using activation loops between pFC and BG. This framework, too, has been used to integrate brain and behavior, capturing the qualitative aspects of behavioral performance and neural activation in the Simon task (Herd, Banich, & O'Reilly, 2006). More generally, this approach has focused on understanding mechanisms of cortical learning and the role of the basal ganglia in motor planning, execution, and inhibition.

One key point of contrast between our DNF model and LEABRA concerns how stimuli and responses are associated. Recall that we used 2-D neural fields that bind stimulus and response information along particular dimensions. These 2-D fields enable Hebbian learning to form fast and flexible task representations on-the-fly. By contrast, learning specific stimulus–response associations can take many trials within connectionist approaches. More critically, dual-task conditions present a binding problem in connectionist-style models because multiple stimuli are simultaneously presented for different tasks. Consequently, the network cannot select which stimulus to bind to which response and forms erroneous associations. Future work will be needed to contrast models like LEABRA and DFT on this front. For example, one consequence of binding stimuli and responses along continuous metric dimensions is that these factors should have interactive effects. That is, the DNF model predicts that the similarity of the stimuli and responses of a task should produce an interaction of factors as information is integrated within 2-D fields rather than additive effects.

The work presented here brings us closer to a general theory of response selection and provides a novel framework for explaining perception–action translation. DFT integrates behavioral and neural dynamics within the same model, moving interpretations of neuroimaging data beyond simply accounting for “where” cognitive processes can be localized toward a cognitively functional account of neural dynamics. Our report represents one of the first instances in which behavioral and hemodynamic data have been simulated in quantitative detail using the same model and parameters (see Jilk et al., 2008). Critically, these modeling efforts effectively addressed two central questions from the dual-task literature, elucidating mechanisms underlying changes in dual-task efficiency over learning.

APPENDIX

In this section, we describe the equations used in the DNF simulations. Table 2 shows the notation used in the equations below. The basic formulation of a 2-D neural system tuned, for instance, to visual and manual dimensions is given by Equation 1 (excitatory layer) and Equation 2 (inhibitory layer). The rate of change in activation in an excitatory cortical field, w, evolves over time, t, at each location in the field along a response dimension, x, and a stimulus dimension, y. The first part of Equations 1 and 2 reflect stabilization terms (−w(x, y, t) and −v(x, y, t)) and the neural resting levels (he and hi). The stabilization terms serve to maintain activation around an attractor state. That is, the rate of change goes in the opposite direction of perturbations away from an attractor. The resting level sets the baseline activation below the activation threshold. The excitatory layer has a further term specifying a stimulus input that is function of the location in the field and the time point in the simulation (s(x, y, t)). A final contribution is from spatially correlated noise (∫ cr(xx′, yy′)ξ(x′, y′, t)dx′dy′). Note that the tau parameters (τe and τi) reflects the timescale at which activation approaches an attractor state.
formula
formula
Table 2. 

Notation

Letter
Meaning
a amplitude/strength parameter 
x,y dimension (x = stimulus, y = response) 
w activation variable for WM layer 
v activation variable for inhibitory layer 
d activation variable for attention units 
Ω activation variable for CoS units 
m activation variable for memory/Hebbian layer 
s stimulus input (Guassian for fields) 
c connection weight function 
g gating function 
t time 
τ timescale parameter 
h resting level parameter 
n number of nodes/generic node 
η number of units along a dimension 
r random contribution 
ξ noise variable 
e excitatory 
i inhibitory 
j Index 1 
k Index 2 
Letter
Meaning
a amplitude/strength parameter 
x,y dimension (x = stimulus, y = response) 
w activation variable for WM layer 
v activation variable for inhibitory layer 
d activation variable for attention units 
Ω activation variable for CoS units 
m activation variable for memory/Hebbian layer 
s stimulus input (Guassian for fields) 
c connection weight function 
g gating function 
t time 
τ timescale parameter 
h resting level parameter 
n number of nodes/generic node 
η number of units along a dimension 
r random contribution 
ξ noise variable 
e excitatory 
i inhibitory 
j Index 1 
k Index 2 
The remainder of the equation specifies the excitatory and inhibitory neural interactions. Neural interactions within a field are determined by the convolution of a sigmoidal threshold function (Equation 3) and Guassian interaction kernel (Equation 4). The beta parameter (β) in Equation 3 specifies the steepness of the transition between weak and strong neural interactions around the activation threshold of 0. The c parameter in Equation 4 specifies the strength of the Gaussian interaction kernel, whereas the sigma parameter (σ) specifies the width of the Guassian interaction kernel. Self-excitation in Equation 1 then is given by the sigmoided output of the w field (g(w(x′, y′, t))) with an excitatory interaction kernel (cvw(xx′, yy′)). Lateral inhibition in this excitatory layer is defined as the sigmoided output form a separate inhibitory field (v; g(v(x′, y′, t))) given by Equation 2. This sigmoided output is convolved with an inhibitory interaction kernel (cwv(xx′, yy′)).
formula
formula
Thus, the excitatory layer has three neural interaction terms for self-excitation, lateral-inhibition, and noise. The contribution of Hebbian memories (∫∫ cwm(xx′, yy′)m(x′, y′, t)) is given by two separate equations reflecting the rise and decay of memories (see Equations 5.15.3). Equation 5.1 defines the build-up of activation when a peak of activation is present in an associated excitatory layer (i.e., w(x, y, t) > 0). The decay of Hebbian memories is driven by Equation 5.3 (i.e., w(x, y, t) < 0). The inhibitory layer has terms for the input from the excitatory layer along with noise. The same equations govern activation with the AV system, except for the parameter differences in Table 3.
formula
formula
formula
Table 3. 

Parameters of the Model Used to Simulate the Data from Dux et al. (2009)

Parameter
Value
σvw 2.00 
avw 0.90 
βvw 
σwv 15.00 
awv 1.80 
cwv_global 0.15 
βwv 
σww 3.00 
aww 2.00 
βww 
ade 2.3 
βde 
adi 2.85 
βdi 0.5 
aΩe 1.5 
βΩe 
adw 0.0372 
βdw 0.5 
awd 3.68 
βwd 0.5 
aΩW 0.75 
βΩW 
ar 0.02 
hi −8 
τi 20, 5 
he −8.19, −9 
τe 160, 40 
hd −10 
τd 80 
hΩ −10 
τΩ 80 
σwm 6.0 
awm 0.13 
τw_build 3.0 × 103 
τw_decay 8.0 × 105 
adm 3.45 
τd_build 3.5 × 104 
τd_decay 1.5 × 109 
aΩm 1.0 
τΩ_build 3.5 × 104 
τΩ_decay 1.5 × 109 
n 
η 101 
Parameter
Value
σvw 2.00 
avw 0.90 
βvw 
σwv 15.00 
awv 1.80 
cwv_global 0.15 
βwv 
σww 3.00 
aww 2.00 
βww 
ade 2.3 
βde 
adi 2.85 
βdi 0.5 
aΩe 1.5 
βΩe 
adw 0.0372 
βdw 0.5 
awd 3.68 
βwd 0.5 
aΩW 0.75 
βΩW 
ar 0.02 
hi −8 
τi 20, 5 
he −8.19, −9 
τe 160, 40 
hd −10 
τd 80 
hΩ −10 
τΩ 80 
σwm 6.0 
awm 0.13 
τw_build 3.0 × 103 
τw_decay 8.0 × 105 
adm 3.45 
τd_build 3.5 × 104 
τd_decay 1.5 × 109 
aΩm 1.0 
τΩ_build 3.5 × 104 
τΩ_decay 1.5 × 109 
n 
η 101 

The only values that differed between the AV and VM fields were τv, hw, and τw. For parameters that are different between the AV and VM systems, the value for the VM field is given first, followed by the AV field.

The dynamics for the Att units is given by Equation 6. This takes the form of discrete neural unit, so activation now only is a function of time. The equation takes the same general form as the equations for the fields presented above. This node is self-excitatory and receives input from the 2-D VM field (ad ∫∫ w(x′, y′, t)dx′dy′). This input from the 2-D VM field is gated by the sigmoided output of the VM CoS unit (Ω; given by Equation 7), such that the Att node receives input from the VM field when the CoS node is subthreshold but receives no input from the VM field when the CoS node is activated above threshold. The Att unit also receives inhibitory input from the Att unit associated with the AV task. The CoS node is also self-excitatory and receives an input from the VM response selection field. The Att and CoS units both receive inputs from their associated 2-D response selection field; however, a critical difference in their dynamics is produced by different beta parameters for the sigmoided output of the 2-D response selection field. For example, the VM field is thresholded with a beta value of 5 for the input to the CoS node (meaning that there is a more abrupt transition between weak and strong neural output around the threshold value of 0, engaging the CoS unit only after a robust peak of activation has formed). The VM field is thresholded with a beta value of 0.5, however, for the input to the Att node (meaning weak activation levels as inputs build-up within the VM field will produce stronger output to the Att unit, engaging the Att before the formation of a peak of activation).
formula
formula
Finally, Equations 8.18.3 define the Hebbian learning process that operates on the Att and CoS units. The equations for the Att and CoS units are the same; thus, only the equations for the Att unit are shown. As with Hebbian learning within the 2-D response selection fields, the rise and decay of Hebbian memories is accounted for with two different equations (Equations 8.2 and 8.3, respectively).
formula
formula
formula

There were two types of inputs given to the model. One set of inputs was a simplification of the instruction procedure. That is, instead of instructing the model through the accumulation of Hebbian memories, we applied localized Gaussian inputs (see Figure 5A). These had a strength of 0.35 and a width along both dimensions of 4 units. The second type of input was a ridge input that reflected the presentation of a stimulus on a given trial. This had a strength of 5.5 for the AV field but was scaled to be 15% stronger for the VM field. The attention nodes were also given an input with a strength of 3.5 when a stimulus was presented to the associated response selection field.

The impulse response function (h(t)) used to simulate hemodynamics from the model is defined by Equation 9 (Deco & Rolls, 2005; n1 = 7.000, t1 = 0.875, n2 = 12.250, t2 = 1.000, a2 = 1.6 × 10−12).
formula
Finally, Equations 10 and 11 define the DFT-LFP from different components of the model.
formula
formula

Acknowledgments

The authors thank Gregor Schöner for critical discussions as we developed the DNF model and the method for simulating hemodynamics. This work was supported by National Science Foundation grant BCS-1029082 awarded to J. P. S. and National Institute of Health grant DA031583 awarded to E. H., A. T. B., T. W., and J. P. S.

Reprint requests should be sent to Aaron T. Buss, Department of Psychology, The University of Iowa, 11 Seashore Hall E, Iowa City, IA 52240, or via e-mail: aaron-buss@uiowa.edu.

REFERENCES

Amari
,
S.
(
1977
).
Dynamics of pattern formation in lateral-inhibition type neural fields.
Biological Cybernetics
,
27
,
77
87
.
Anderson
,
J. R.
,
Bothell
,
D.
,
Byrne
,
M. D.
,
Douglass
,
S.
,
Lebiere
,
C.
, &
Qin
,
Y.
(
2004
).
An integrated theory of the mind.
Psychological Review
,
111
,
1036
1060
.
Anderson
,
J. R.
,
Qin
,
Y.
,
Jung
,
K.-J.
, &
Carter
,
C. S.
(
2007
).
Information-processing modules and their relative modality specificity.
Cognitive Psychology
,
54
,
185
217
.
Anderson
,
J. R.
,
Taatgen
,
N. A.
, &
Byrne
,
M. D.
(
2005
).
Learning to achieve perfect timesharing: Architectural implications of Hazeltine, Teague, and Ivry (2002).
Journal of Experimental Psychology: Human Perception and Performance
,
31
,
749
761
.
Asaad
,
W. F.
,
Rainer
,
G.
, &
Miller
,
E. K.
(
2000
).
Task-specific neural activity in the primate prefrontal cortex.
Journal of Neurophysiology
,
84
,
451
459
.
Ashby
,
F. G.
, &
Waldschmidt
,
J. G.
(
2008
).
Fitting computational models to fMRI data.
Behavior Research Methods
,
40
,
713
721
.
Bausenhart
,
K. M.
,
Rolke
,
B.
,
Hackley
,
S. A.
, &
Ulrich
,
R.
(
2006
).
The locus of temporal preparation effects: Evidence from the psychological refractory period paradigm.
Psychonomic Bulletin & Review
,
13
,
536
542
.
Buss
,
A. T.
, &
Spencer
,
J. P.
(
in press
).
The emergent executive: A dynamic neural field theory of the development of executive function.
Monographs of the Society for Research in Child Development
.
Cohen
,
M. A.
,
Horowitz
,
T. S.
, &
Wolfe
,
J. M.
(
2009
).
Auditory recognition memory is inferior to visual recognition memory.
Proceedings of the National Academy of Sciences, U.S.A.
,
106
,
6008
6010
.
Damasio
,
A. R.
(
1989
).
Time-locked multiregional retroactivation: A systems-level proposal for the neural substrates of recall and recognition.
Cognition
,
33
,
25
62
.
Davis
,
T.
,
Love
,
B. C.
, &
Preston
,
A. R.
(
2012
).
Learning the exception to the rule: Model-based fMRI reveals specialized representations for surprising category members.
Cerebral Cortex
,
22
,
260
273
.
Deco
,
G.
, &
Rolls
,
E. T.
(
2005
).
Attention, short-term memory, and action selection: A unifying theory.
Progress in Neurobiology
,
76
,
236
256
.
Deco
,
G.
,
Rolls
,
E. T.
, &
Horwitz
,
B.
(
2004
).
“What” and “where” in visual working memory: A computational neurodynamical perspective for integrating fMRI and single-neuron data.
Journal of Cognitive Neuroscience
,
16
,
683
701
.
Dux
,
P. E.
,
Tombu
,
M. N.
,
Harrison
,
S.
,
Rogers
,
B. P.
,
Tong
,
F.
, &
Marois
,
R.
(
2009
).
Training improves multitasking performance by increasing the speed of information processing in human prefrontal cortex.
Neuron
,
63
,
127
138
.
Erickson
,
K. I.
,
Colcombe
,
S. J.
,
Wadhwa
,
R.
,
Bherer
,
L.
,
Peterson
,
M. S.
,
Scalf
,
P. E.
,
et al
(
2007
).
Training-induced functional activation changes in dual-task processing: An fMRI study.
Cerebral Cortex
,
17
,
192
204
.
Erlhagen
,
W.
, &
Schöner
,
G.
(
2002
).
Dynamic field theory of movement preparation.
Psychological Review
,
109
,
545
572
.
Erlhange
,
W.
,
Bastian
,
A.
,
Jancke
,
D.
,
Riehle
,
A.
, &
Schoner
,
G.
(
1999
).
The distribution of neuronal population activation (DPA) as a tool to study interaction and integration of cortical representation.
Journal of Neuroscience Methods
,
94
,
53
66
.
Faubel
,
C.
,
Dineva
,
E.
, &
Bicho
,
E.
(
in press
).
Embodied neural dynamics.
In G. Schoner, J. P. Spencer, & the DFT Research Group (Eds.)
,
Dynamic thinking—A primer on dynamic field theory.
New York
:
Oxford University Press
.
Faubel
,
C.
, &
Schöner
,
G.
(
2008
).
Learning to recognize objects on the fly: A neurally based dynamic field approach.
Neural Networks
,
21
,
562
576
.
Fuster
,
J. M.
(
1989
).
The prefrontal cortex.
New York
:
Raven
.
Gross
,
C. G.
,
Rocha-Miranda
,
C. E.
, &
Bender
,
D. B.
(
1972
).
Visual properties of neurons in inferotemporal cortex of the macaque.
Journal of Neurophysiology
,
35
,
96
111
.
Hazeltine
,
E.
(
2005
).
Response-response compatibility during bimanual movements: Evidence for the conceptual coding of action.
Psychonomic Bulletin & Review
,
12
,
682
688
.
Hazeltine
,
E.
,
Ruthruff
,
E.
, &
Remington
,
R. W.
(
2006
).
The role of input and output modality pairings in dual-task performance: Evidence for content-dependent central interference.
Cognitive Psychology
,
52
,
291
345
.
Hazeltine
,
E.
,
Teague
,
D.
, &
Ivry
,
R. B.
(
2002
).
Simultaneous dual-task performance reveals parallel response selection after practice.
Journal of Experimental Psychology: Human Perception and Performance
,
28
,
527
545
.
Hazy
,
T. E.
,
Frank
,
M. J.
, &
O'Reilly
,
R. C.
(
2007
).
Towards an executive without a homunculus: Computational models of the prefrontal cortex/basal ganglia system.
Philosophical Transactions of the Royal Society of London, Series B, Biological Sciences
,
362
,
1601
1613
.
Herd
,
S. A.
,
Banich
,
M. T.
, &
O'Reilly
,
R. C.
(
2006
).
Neural mechanisms of cognitive control: An integrative model of Stroop task performance and fMRI data.
Journal of Cognitive Neuroscience
,
18
,
22
32
.
Jancke
,
D.
,
Erlhagen
,
W.
,
Dinse
,
H. R.
,
Akhavan
,
A. C.
,
Giese
,
M.
,
Steinhage
,
A.
,
et al
(
1999
).
Parametric population representation of retinal location: Neuronal interaction dynamics in cat primary visual cortex.
The Journal of Neuroscience
,
19
,
9016
9028
.
Jilk
,
D.
,
Lebiere
,
C.
,
O'Reilly
,
R.
, &
Anderson
,
J.
(
2008
).
SAL: An explicitly pluralistic cognitive architecture.
Journal of Experimental & Theoretical Artificial Intelligence
,
20
,
197
218
.
Johnson
,
J. S.
,
Spencer
,
J. P.
, &
Schöner
,
G.
(
2008
).
Moving to higher ground: The dynamic field theory and the dynamics of visual cognition.
New Ideas in Psychology
,
26
,
227
251
.
Jonides
,
J.
(
2004
).
How does practice makes perfect?
Nature Neuroscience
,
7
,
10
11
.
Kelly
,
A. M. C.
, &
Garavan
,
H.
(
2005
).
Human functional neuroimaging of brain changes associated with practice.
Cerebral Cortex
,
15
,
1089
1102
.
Lipinski
,
J.
,
Schneegans
,
S.
,
Sandamirskaya
,
Y.
,
Spencer
,
J. P.
, &
Schöner
,
G.
(
2012
).
A neurobehavioral model of flexible spatial language behaviors.
Journal of Experimental Psychology: Learning, Memory, and Cognition
,
38
,
1490
1511
.
Logothetis
,
N. K.
,
Pauls
,
J.
,
Augath
,
M.
,
Trinath
,
T.
, &
Oeltermann
,
A.
(
2001
).
Neurophysiological investigation of the basis of the fMRI signal.
Nature
,
412
,
150
157
.
McDowell
,
K.
,
Jeka
,
J. J.
,
Schoner
,
G.
, &
Hatfield
,
B. D.
(
2002
).
Behavioral and electrophysiological evidence of an interaction between probability and task metrics in movement preparation.
Experimental Brain Research
,
144
,
303
313
.
Meyer
,
D. E.
, &
Kieras
,
D. E.
(
1997
).
A computational theory of executive cognitive processes and multiple-task performance: Part 2. Accounts of psychological refractory-period phenomena.
Psychological Review
,
104
,
749
791
.
Mishkin
,
M.
,
Ungerleider
,
L. G.
, &
Macko
,
K. A.
(
1983
).
Object vision and spatial vision: Two cortical pathways.
Trends in Neurosciences
,
6
,
414
417
.
O'Reilly
,
R. C.
,
Noelle
,
D. C.
,
Braver
,
T. S.
, &
Cohen
,
J. D.
(
2002
).
Prefrontal cortex and dynamic categorization tasks: Representational organization and neuromodulatory control.
Cerebral Cortex
,
12
,
246
257
.
Pashler
,
H.
(
1994
).
Dual-task interference in simple tasks: Data and theory.
Psychological Bulletin
,
116
,
220
244
.
Petersen
,
S. E.
,
van Mier
,
H.
,
Fiez
,
J. A.
, &
Raichle
,
M. E.
(
1998
).
The effects of practice on the functional anatomy of task performance.
Proceedings of the National Academy of Sciences, U.S.A.
,
95
,
853
860
.
Ruthruff
,
E.
,
Johnston
,
J. C.
,
Van Selst
,
M.
,
Whitsell
,
S.
, &
Remington
,
R.
(
2003
).
Vanishing dual-task interference after practice: Has the bottleneck been eliminated or is it merely latent?
Journal of Experimental Psychology: Human Perception and Performance
,
29
,
280
289
.
Samuelson
,
L. K.
,
Smith
,
L. B.
,
Perry
,
L. K.
, &
Spencer
,
J. P.
(
2011
).
Grounding word learning in space.
PloS One
,
6
,
e28095
.
Sandamirskaya
,
Y.
, &
Schöner
,
G.
(
2010
).
An embodied account of serial order: How instabilities drive sequence generation.
Neural Networks: The Official Journal of the International Neural Network Society
,
23
,
1164
1179
.
Schoner
,
G.
, &
Schutte
,
A.
(
in press
).
Introduction to dynamic field theory.
In G. Schoner, J. P. Spencer, & the DFT Research Group (Eds.)
,
Dynamic thinking—A primer on dynamic field theory.
New York
:
Oxford University Press
.
Schumacher
,
E. H.
,
Seymour
,
T. L.
,
Glass
,
J. M.
,
Fencsik
,
D. E.
,
Lauber
,
E. J.
,
Kieras
,
D. E.
,
et al
(
2001
).
Virtually perfect time sharing in dual-task performance: Uncorking the central cognitive bottleneck.
Psychological Science
,
12
,
101
108
.
Searle
,
J. R.
(
1980
).
The intentionality of intention and action.
Cognitive Science
,
4
,
47
70
.
Spencer
,
J. P.
,
Barich
,
K.
,
Goldberg
,
J.
, &
Perone
,
S.
(
2012
).
Behavioral dynamics and neural grounding of a dynamic field theory of multi-object tracking.
Journal of Integrative Neuroscience
,
11
,
339
362
.
Spencer
,
J. P.
,
Schneegans
,
S.
, &
Schoner
,
G.
(
in press
).
Integrating “what” and “where”: Visual working memory for objects in a scene.
In G. Schoner, J. P. Spencer, & the DFT Research Group (Eds.)
,
Dynamic thinking—A primer on dynamic field theory.
New York
:
Oxford University Press
.
Tombu
,
M.
, &
Jolicoeur
,
P.
(
2004
).
Virtually no evidence for virtually perfect time-sharing.
Journal of Experimental Psychology: Human Perception and Performance
,
30
,
795
810
.
Van Selst
,
M.
,
Ruthruff
,
E.
, &
Johnston
,
J. C.
(
1999
).
Can practice eliminate the psychological refractory period effect?
Journal of Experimental Psychology: Human Perception and Performance
,
25
,
1268
1283
.
White
,
C. N.
, &
Poldrack
,
R. A.
(
2013
).
Using fMRI to constrain theories of cognition.
Perspectives on Psychological Science
,
8
,
79
83
.