Abstract

Many cognitive and computational models have been proposed to help understand working memory. In this article, we present a simulation study of cortical processing of visual objects during several working memory tasks using an extended version of a previously constructed large-scale neural model [Tagamets, M. A., & Horwitz, B. Integrating electrophysiological and anatomical experimental data to create a large-scale model that simulates a delayed match-to-sample human brain imaging study. Cerebral Cortex, 8, 310–320, 1998]. The original model consisted of arrays of Wilson–Cowan type of neuronal populations representing primary and secondary visual cortices, inferotemporal (IT) cortex, and pFC. We added a module representing entorhinal cortex, which functions as a gating module. We successfully implemented multiple working memory tasks using the same model and produced neuronal patterns in visual cortex, IT cortex, and pFC that match experimental findings. These working memory tasks can include distractor stimuli or can require that multiple items be retained in mind during a delay period (Sternberg's task). Besides electrophysiology data and behavioral data, we also generated fMRI BOLD time series from our simulation. Our results support the involvement of IT cortex in working memory maintenance and suggest the cortical architecture underlying the neural mechanisms mediating particular working memory tasks. Furthermore, we noticed that, during simulations of memorizing a list of objects, the first and last items in the sequence were recalled best, which may implicate the neural mechanism behind this important psychological effect (i.e., the primacy and recency effect).

INTRODUCTION

The past three decades have witnessed an explosion of studies on memory in both humans and other primates. pFC has been considered to play a crucial role in the maintenance of working memory, the cognitive process that is involved in the transient holding and manipulating of information. A large number of nonhuman primate studies using single-neuron recordings (Miller, Erickson, & Desimone, 1996; Funahashi, Bruce, & Goldman-Rakic, 1989; Fuster, Bauer, & Jervey, 1982) and lesions (Levy & Goldman-Rakic, 1999) have supported the involvement of pFC in working-memory-related processes. Human brain imaging studies (Riley & Constantinidis, 2015; Husain, Tagamets, Fromm, Braun, & Horwitz, 2004; Courtney, Petit, Maisog, Ungerleider, & Haxby, 1998; D'Esposito et al., 1995; Haxby, Ungerleider, Horwitz, Rapoport, & Grady, 1995), using PET and fMRI, have also revealed the crucial role that pFC plays in both object and spatial working memory.

Working memory tasks, such as the delayed match-to-sample (DMS) task and Sternberg's recognition task (Sternberg, 1966, 1969), have been extensively used in the study of the behavior of neurons and microcircuits related to working memory maintenance in pFC (Schon, Ross, Hasselmo, & Stern, 2013; Funahashi, Inoue, & Kubota, 1993; Fuster & Alexander, 1971, 1973; Kubota & Niki, 1971). However, the underlying neurobiological mechanisms for the brain activations seen in these studies and the interactions between pFC and other brain regions, such as inferotemporal (IT) cortex, to support working memory operations are not well understood.

We and others have argued that large-scale neural network models can be powerful tools for addressing such issues (Chaudhuri, Knoblauch, Gariel, Kennedy, & Wang, 2015; Garagnani, Wennekers, & Pulvermuller, 2008; Gisiger & Kerszberg, 2006; Corchs & Deco, 2002; Tagamets & Horwitz, 1998). To understand better the neural substrate of human imaging studies using PET and fMRI, Tagamets and Horwitz proposed a large-scale neural model (LSNM) based on the visual ventral pathway that simulated a DMS task for objects (Horwitz & Tagamets, 1999; Tagamets & Horwitz, 1998). The model was later updated for TMS, structural equation modeling, fMRI, and magnetoencephalography (MEG) simulations (Horwitz & Banerjee, 2012; Kim & Horwitz, 2009; Husain et al., 2002). A similar model was also developed to perform auditory tasks for fMRI simulations (Husain et al., 2004) based on the hypothesis that the sensory cortices involved in visual and auditory object processing perform similar operations but act on different features. These models successfully simulated cortical neuronal activities and functional brain imaging data that generally agree with experimental results. However, a number of limitations of the original models exist, including that (1) the models could not handle distractors or multiple stimuli, (2) the IT neural activity during the delay period did not always match the experimental findings in nonhuman electrophysiology study in which the IT area showed higher-than-baseline activity during the delay period (Fuster et al., 1982), and (3) the rest of the brain was not included in the model.

In this article, we will present an LSNM that can perform multiple STM tasks for novel objects, with the simulated neuronal behaviors and simulated fMRI patterns matching experimental data. We modified and expanded the original model by adding one module representing entorhinal cortex (EC) to perform a gating mechanism. We successfully simulated three versions of DMS tasks with or without distractors and a variation of Sternberg's recognition task that required the model to hold multiple items in working memory. A primacy/recency effect was observed during the simulations of memorizing a list of objects, that is, the first and last items in the sequence were recalled best, which may implicate the neural mechanisms behind this important psychological effect. Moreover, we have now improved the match between simulated and experimental activity and observed rich neuronal behaviors during the delay period in the module representing IT cortex. Finally, we have placed our model in a whole-brain connectome framework (Ulloa & Horwitz, 2016). In this article, we will restrict our attention to the visual processing model.

METHODS

Brief Review of Visual Object Processing Model

The visual object processing model developed by Tagamets and Horwitz (Horwitz & Tagamets, 1999; Tagamets & Horwitz, 1998) consisted of interconnected neuronal populations representing the cortical ventral pathway found to process primarily the features of a visual object (Haxby et al., 1991; Mishkin & Ungerleider, 1982; see Figure 1 of Ulloa & Horwitz, 2016). Shape is the feature used in the model to characterize a visual object. Model neurons in modules representing the early visual cortex were assumed to be orientation selective (for simplicity, horizontal and vertical orientations were used). Beginning in striate visual cortex, the ventral processing pathway extends into the IT gyrus and projects into ventrolateral pFC. The modules that comprise the visual model include ones representing primary and secondary visual cortex (V1/V2), area V4, anterior IT cortex, and pFC. Each of these regions contains one or more neural populations with different functional attributes (discussed below). The response properties of the simulated neuronal populations were based on known monkey neural electrophysiological data (e.g., Funahashi, Bruce, & Goldman-Rakic, 1990). An important assumption for the visual model, inferred from such experimental data, was that the spatial receptive field of neurons increased along the ventral processing pathway.

Each neuronal population in our model consisted of 81 microcircuits, each representing a cortical column. As shown in Figure 1A, the model employed modified Wilson–Cowan units (an interacting excitatory and inhibitory pair of elements) as the microcircuit (Tagamets & Horwitz, 1998; Wilson & Cowan, 1972). The input synaptic activity to each microcircuit can also be evaluated, and combinations of this activity were related to the fMRI or MEG/EEG signals via a forward model. In this article, we will only consider simulated fMRI (Horwitz & Tagamets, 1999).

Figure 1. 

(A) Structure of a Wilson–Cowan microcircuit, which can be considered as a simplified representation of a cortical column. Each microcircuit consists of an excitatory element and an inhibitory element with the excitatory element corresponding to the pyramidal neuronal population in a column and the inhibitory element corresponding to the inhibitory interneurons. (B) The network structure of the modified visual LSNM. Compared with the original LSNM structure (Ulloa & Horwitz, 2016; Tagamets & Horwitz, 1998), (1) a gating module (GU) has been added, which is tentatively located in EC, and (2) multiple sets of working memory modules (D1 and D2 in pFC) are used, instead of one set of D1 and D2 units in the original model. (C) The EC and additional working memory modules are designed to act as a gate between IT cortex and pFC. Multiple groups of entorhinal neurons and pFC neurons are incorporated to hold multiple items in STM. The entorhinal neurons competitively inhibit each other so that a group of gating neurons will be activated when a stimulus comes in and inhibits other groups of gating neurons. Once the item is stored, an inhibitory feedback from pFC to EC will suppress the active gating neurons and release other gating neurons so that the remaining gating neurons are ready for new stimuli.

Figure 1. 

(A) Structure of a Wilson–Cowan microcircuit, which can be considered as a simplified representation of a cortical column. Each microcircuit consists of an excitatory element and an inhibitory element with the excitatory element corresponding to the pyramidal neuronal population in a column and the inhibitory element corresponding to the inhibitory interneurons. (B) The network structure of the modified visual LSNM. Compared with the original LSNM structure (Ulloa & Horwitz, 2016; Tagamets & Horwitz, 1998), (1) a gating module (GU) has been added, which is tentatively located in EC, and (2) multiple sets of working memory modules (D1 and D2 in pFC) are used, instead of one set of D1 and D2 units in the original model. (C) The EC and additional working memory modules are designed to act as a gate between IT cortex and pFC. Multiple groups of entorhinal neurons and pFC neurons are incorporated to hold multiple items in STM. The entorhinal neurons competitively inhibit each other so that a group of gating neurons will be activated when a stimulus comes in and inhibits other groups of gating neurons. Once the item is stored, an inhibitory feedback from pFC to EC will suppress the active gating neurons and release other gating neurons so that the remaining gating neurons are ready for new stimuli.

This visual LSNM was designed to perform a short-term recognition memory DMS task. During each trial of the task, a stimulus S1 is presented for a certain amount of time, followed by a delay period in which S1 has to be kept in STM. When a second stimulus (S2) is presented, the model has to respond as to whether S2 matches S1. The model also performs a control task: passive perception of the stimuli. Multiple trials of the active and passive tasks constitute a simulated functional neuroimaging study.

Ulloa and Horwitz (2016) embedded the visual LSNM into a whole-brain framework using The Virtual Brain (TVB) software package (Sanz Leon et al., 2013). TVB is a simulator that combines (i) white matter structural connections among brain regions to simulate long-range connections, (ii) a neuronal population model to simulate local brain activity, and forward models that convert simulated neural activity into simulated functional neuroimaging data (i.e., fMRI or EEG/MEG). The TVB framework is helpful for simulating functional and effective connectivity neuroimaging data (see Ulloa & Horwitz, 2016), and the interaction between our model and TVB enables us to explore how task and intrinsic activities interact (Ulloa & Horwitz, in preparation). In the current article, TVB neurons provide neural noise to the embedded LSNM. The structural connectome we employ is that due to Hagmann et al. (2008), which is composed of 998 ROIs, and the simulated neuronal microcircuits at each TVB node are Wilson–Cowan units.

The Structural Network

The structural network of the modified visual model is shown in Figure 1B. Each module of the modified network is explained in detail in the following. As detailed in the original article by Tagamets and Horwitz (1998), most of the connections are hand tuned. However, the connections between V4 and IT were learned by means of a Hebbian algorithm.

V1/V2

In the model, the early visual areas V1 and V2 are combined and are designed to consist of orientation selective units (for simplicity, we have only employed horizontal selective units and vertical selective units). Single-neuron recording experiments in primates have confirmed that neurons exist in both V1 and V2 areas that respond preferentially to visual features such as line orientation, edges, and colors (Roe & Ts'o, 1995; Peterhans & von der Heydt, 1993; Hubel, Wiesel, & Stryker, 1977). This formulation is unchanged from the original Tagamets and Horwitz (1998) model.

V4

The V4 area is designed as a continuation of the shape processing pathway and consists of three populations of units: horizontal selective units, vertical selective units, and corner selective units. They are constructed to have an increased spatial receptive field relative to V1/V2; that is, they respond to longer horizontal and vertical line segments and also corners formed by adjacent pairs of horizontal and vertical lines. Experimental studies provide the basis for this design; neurons in area V4 share similar properties with earlier areas but appear to encode more complex properties of shape (Gallant, Braun, & Van Essen, 1993; Desimone & Schein, 1987). As with V1/V2, this module is the same as in the original model. We did add one new item, however; there is now feedback connectivity from V4 to V1/V2.

IT

The next processing module of the model corresponds to inferior temporal cortex and is denoted by IT. The IT module functions as a feature integrator and generates the initial rough representation of a percept. In single-neuron recordings, as we mentioned in the introduction, IT areas exhibit various active behaviors during the short delay periods that might be relevant to visual feature-selective activity and visual information maintenance (Ranganath & D'Esposito, 2005; Petrides, 2000; Miyashita, 1988; Horel, Pytko-Joiner, Voytko, & Salsbury, 1987; Fuster et al., 1982). IT was also a module in the original model.

EC

In our new model, we have added a new module—EC—to serve as a gate between IT cortex and pFC. We designed the gating mechanism by incorporating several groups of neurons in the EC that competitively inhibit one another (see Figure 1C). The purpose of such a mechanism is to avoid the working memory of one stimulus being overwritten by later incoming stimuli. We tentatively assigned the EC based on results of a series of empirical findings. The involvement of EC in working memory encoding has been supported by multiple experimental studies (for a review, see Lech & Suchan, 2014), although the actual neural mechanisms remain unclear. Divergent projections from the anterior IT area to the EC have been verified by anatomical studies in monkeys (Saleem & Tanaka, 1996). Some recent evidence suggests that the outputs of the medial pFC are sent back to the lateral EC and perirhinal cortex (Navawongse & Eichenbaum, 2013). In our design, a group of gating neurons will be activated when a stimulus comes in and inhibits other groups of gating neurons. Once the item is stored in this working memory buffer, an inhibitory feedback from pFC to EC will suppress the active gating neurons and release other gating neurons so that the remaining gating neurons are ready for new stimuli. By such a design, we are assuming that each group of entorhinal gating neurons could be used only once during a task trial.

Prefrontal Area

Model neurons in the pFC module, in an STM task condition, can be delineated into four types based on experimental data acquired by Funahashi et al. (1990). In our model (see Figure 1B), the submodule FS contains cue-sensitive units that in general reflect the activities in the IT module. D1 and D2 submodules form the STM units by exciting one another during the delay period. D1 is active only during the delay period, and D2 is active during stimuli presentations and the delay period. In our modified model, we now have multiple sets of D1 and D2 submodules built into the model such that holding more than one item in STM is possible (see Figure 1C). R serves as a response module (output). It receives information from D1 (containing a representation of a stimulus being kept in STM) and FS (containing the temporary representation of the stimulus just presented). FS or D1 alone cannot activate R. When the incoming stimulus matches an item in working memory, FS and D1 together can activate R. Note also that we assume that there are a limited number of gating units and a similar limited number of D1–D2 units, because empirical studies indicate that only a limited number of items can be simultaneously kept in STM (e.g., the so-called 7 ± 2 [Miller, 1956]; others have proposed a more limited capacity such as 3 or 4 [Cowan, 2001]). For computational simplicity, in this article, we will employ no more than three items.

We first found the hypothetical ROIs corresponding to each module in our LSNM and the connected nodes in Hagmann's connectome. Then, we embedded our revised model of microcircuits and network structure into the connectome. We ran the simulations using our in-house simulator in parallel with Hagmann's connectome using TVB software (Sanz Leon et al., 2013). Figure 2 shows the embedded model in the Hagmann's connectome. Overall, the LSNM embedded in TVB was able to perform the DMS task, generated simulated neural activities in the various brain regions that match empirical data from nonhuman preparations, and produced simulated functional neuroimaging data that generally agree with human experimental findings (see Ulloa & Horwitz, 2016, for details).

Figure 2. 

Embedded model in Hagmann's connectome (Hagmann et al., 2008). We first found hypothetical locations for our model's ROIs and the connected nodes in the connectome (small dots connected to ROIs). We embedded our model of microcircuits and network structure into the structural connectome model of Hagmann et al. (2008). See Ulloa and Horwitz (2016) for details.

Figure 2. 

Embedded model in Hagmann's connectome (Hagmann et al., 2008). We first found hypothetical locations for our model's ROIs and the connected nodes in the connectome (small dots connected to ROIs). We embedded our model of microcircuits and network structure into the structural connectome model of Hagmann et al. (2008). See Ulloa and Horwitz (2016) for details.

Although arbitrary, a task specification module, which is located in the superior frontal gyrus of TVB model, gives instructions to the model concerning what task is going to be performed: whether only the first stimulus should be remembered or a list of items should be remembered. This module provides a low-level, diffuse incoming activity to the D2 module in the prefrontal area, which can be interpreted as an attention level. The attention level/task parameter can be modulated and is specified before each trial in a simulation. High attention will ensure a (better) working memory for the presenting stimulus. When the attention level is low, the working memory modules are not able to hold a stimulus throughout the delay period. Note that the mutual inhibition within the EC module does not depend explicitly on the level of attention.

The Talairach coordinates (Talairach & Tournoux, 1988) and the closest node in Hagmann's connectome for each of the LSNM modules discussed above were identified (see Table 1), based on visual experimental findings (Haxby et al., 1991). As to the prefrontal module, which contains four submodules (FS, D1, D2, and R), we used the Talairach coordinates of pFC in Haxby et al. (1995) for the D1 submodule and assigned the locations of adjacent nodes for the rest of the submodules (FS, D2, R; see Table 1). This arrangement is due to the fact, as mentioned above, that the four types of neuronal populations were based on the experimental findings in monkey pFC during a delayed response task (Funahashi et al., 1990). It is not known if these four neuronal types were found in separate anatomical locations in pFC or were found in the same brain region.

Table 1. 

The Talairach Coordinates (Talairach & Tournoux, 1988) and the Closest Node in the Hagmann's Connectome (Hagmann et al., 2008) Corresponding to Visual LSNM Modules

Modules Talairach Location Source Host Connectome Node 
V1/V2 (18, −88, 8) Haxby et al., 1995  (14, −86, 7) 
V4 (30, −72, −12) Haxby et al., 1995  (33, −70, −7) 
IT (28, −36, −8) Haxby et al., 1995  (31, −39, −6) 
EC (25, −12, −25) Hagmann et al., 2008  (25, −12, −25) 
FS Location selected for illustrative purposes (47, 19, 9) 
D1 (42, 26, 20) Haxby et al., 1995  (43, 29, 21) 
D2 Location selected for illustrative purposes (42, 39, 2) 
Location selected for illustrative purposes (29, 25, 40) 
Modules Talairach Location Source Host Connectome Node 
V1/V2 (18, −88, 8) Haxby et al., 1995  (14, −86, 7) 
V4 (30, −72, −12) Haxby et al., 1995  (33, −70, −7) 
IT (28, −36, −8) Haxby et al., 1995  (31, −39, −6) 
EC (25, −12, −25) Hagmann et al., 2008  (25, −12, −25) 
FS Location selected for illustrative purposes (47, 19, 9) 
D1 (42, 26, 20) Haxby et al., 1995  (43, 29, 21) 
D2 Location selected for illustrative purposes (42, 39, 2) 
Location selected for illustrative purposes (29, 25, 40) 

Note that the locations of FS, D1, D2, and R are not explicitly known (see text) and were chosen only to demonstrate validity of the method.

Simulated Experiments

We use the extended model to perform a number of simulated experiments that can include not only one stimulus but also others, some of which can be considered to be distractors. The complete set of simulated experiments is discussed below.

Experiment 1: Single Stimulus

We first displayed a single stimulus to the model as a test to observe the responses of different modules of the model to a transient visual input. No response is required. The attention/task parameter is set to a high value (0.3).

Experiment 2: DMS Task

This experiment implemented the original DMS task to demonstrate that the new model (with an added node—the EC) continues to perform the DMS task and gives the same results as the original model (Ulloa & Horwitz, 2016). One typical DMS trial consists of the presentation of a stimulus, an ensuing delay period, and a presentation of a probe (the same stimulus or a new stimulus), and at the end of it, the simulated participants need to decide whether the probe is the same as the first stimulus presented (see Figure 3A). The attention/task parameter is set to high (0.3) during a trial.

Figure 3. 

(A) The timeline and task parameter/attention level of a single DMS trial. The simulated participants' task is to identify whether the probe is a match with the first stimulus. (B) The timeline for a single trial of a DMS task with distractors. The simulated participants need to ignore the two intervening distractors and only respond to the probe. (C) The timeline for a single trial of an “ABBA” task, that is, a DMS task with two repeated distractors. (D) The timeline for a single trial of Sternberg's recognition task. The simulated participants are shown a list of stimuli, and their task is to decide, after a delay, whether the probe is a match with any stimulus in the list.

Figure 3. 

(A) The timeline and task parameter/attention level of a single DMS trial. The simulated participants' task is to identify whether the probe is a match with the first stimulus. (B) The timeline for a single trial of a DMS task with distractors. The simulated participants need to ignore the two intervening distractors and only respond to the probe. (C) The timeline for a single trial of an “ABBA” task, that is, a DMS task with two repeated distractors. (D) The timeline for a single trial of Sternberg's recognition task. The simulated participants are shown a list of stimuli, and their task is to decide, after a delay, whether the probe is a match with any stimulus in the list.

Experiment 3: DMS Task with Distractors

We implemented a version of DMS task with intervening distractors. The simulated participants were shown two distractors before the probe was displayed (see Figure 3B). The attention/task parameter is set to high (0.3) for the first stimulus and decreased to low (0.05) after the presentation of the distractors.

Experiment 4: “ABBA” Task

A special version of the distractor task (the “ABBA” task) is used. The “ABBA” task was employed by Miller, Gochin, and Gross (1993) in monkey electrophysiology experiments. The model is supposed to hold its response when repeated distractors (“B”) are shown and to respond only to the matched stimulus (“A”; see Figure 3C).

Experiment 5: Sternberg's Recognition Task

A variant of Sternberg's recognition task (Sternberg, 1966, 1969) was used. On each trial of the simulation, three stimuli were presented sequentially, followed by a delay period and then a probe. The participants' task was to decide whether the probe was a match to any of the three stimuli presented earlier (see Figure 3D). The Sternberg paradigm with visual objects has been used in many studies and thus allows us to compare our simulated results with experimental results.

Simulating Electrical Activity and fMRI Activity

We simulated the electrical activity and fMRI activity of the model as our measure while implementing the tasks discussed above. The electrical activity of one simulated neuronal unit, a modified Wilson–Cowan configuration, is given by the following equations:
dEitdt=Δ11+eKEwEEEit+wIEIit+iniEtϕE+NtδEit
and
dIitdt=Δ11+eKIwEIEit+iniItϕI+NtδIit
where Δ is the rate of change; δ is the decay rate; KE and KI are gain constants; wEE, wIE, and wEI are the connectivity weights within one neuronal unit; ϕE and ϕI are the input threshold; and N(t) is the noise. iniE(t) and iniI(t) are the incoming inputs from other nodes. iniE(t) is given by
iniEt=jwjiEEjt+jwjiIIjt+jcjizjiCCjt
where wkiE and wkiI are the weights for connections from the excitatory (E) and inhibitory (I) elements of jth LSNM unit to the excitatory element of ith LSNM unit, Cj is the electrical activity of the connectome excitatory unit j connected to LSNM unit i, and zjiC is the connection weight. cji is a coupling term obtained by the Gaussian pseudorandom number generator of Python. iniI (t) is given by
iniIt=kwkiEEkt+kwkiIIkt
where wkiE and wkiI are the weights for connections from the excitatory (E) and inhibitory (I) elements of kth LSNM unit to the excitatory element of ith LSNM unit.
The integrated synaptic activity is computed before computing fMRI BOLD activity by spatially integrating over each LSNM module and temporally integrating over 50 msec (Horwitz & Tagamets, 1999):
rSYN=INit
where INi (t) is the sum of absolute values of inputs to the excitatory and inhibitory elements of unit i, at time t:
INit=wEEEit+wEIEit+wIEIit+k,iwkiEkt
The last term is the sum of synaptic connections from all other LSNM units and connectome nodes to the ith unit in LSNM.

In simulating an fMRI study, the model alternately implements a block of DMS task trials (three trials) and a block of control task trials (three trials). The control task used degraded shapes, and each trial of the control task followed the design of the DMS task in Experiment 2, except that the attention/task parameter is set to a low value. We first computed the integrated synaptic activity for select ROIs (Ulloa & Horwitz, 2016). Using the integrated synaptic activity of ROIs as the input to the fMRI BOLD balloon model of hemodynamic response (Ulloa & Horwitz, 2016; Stephan, Weiskopf, Drysdale, Robinson, & Friston, 2007), we calculated the simulated fMRI signal time series for all our ROIs and then downsampled the time series to correspond to a repetition time value of 2 sec.

A top–down task control signal is also used before each trial. The top–down task control signal informs the model that the trial is a DMS task, a DMS task with distractors, an “ABBA” task in which only the first stimulus is the target to be remembered, or a Sternberg's recognition task in which there are multiple targets to remember. The top–down control does not change the network structure; it only controls the attention module so as to apply high attention to targets and low attention to distractors.

In each of the tasks, the simulated stimulus was on for 2 sec (one time step in the model is considered to have a duration of 5 msec) followed by a 4-sec delay period. After each trial, the model was reset in the intertrial interval. When performing the tasks, we varied the connectivity weights between brain regions by slight amounts to create multiple “participants” (see Ulloa & Horwitz, 2016). In both the DMS task and the Sternberg's recognition task, the short delay periods between the presentations of stimuli and the probe are the main elements that make the two tasks tests of STM.

RESULTS

Response to a Single Stimulus

Figure 4 shows the responses of the different modules of the model when a single visual input (a shape composed of horizontal and vertical line segments) was displayed. The attention level was set to high. Each module of the model exhibited proper behaviors in the simulation. Early visual cortex (V1/V2) responded quickly to the stimulus and displayed a sharp decrease in activity when the stimulus disappeared. As the visual input propagated deeper into the network, the average activity displayed slower and smoother responses. For instance, the average activity of working memory modules (D1, D2) slowly climbed during the presentation of the stimulus and displayed persistent activity in the delay period when the stimulus disappeared. The response module (R) showed only noise because only one stimulus was presented.

Figure 4. 

Response of the model to a single stimulus. One stimulus is shown to the model for 2 sec, followed by a 4-sec delay period before resetting. The vertical axis is the normalized mean neuronal activity (i.e., normalized firing rate) in the different modules.

Figure 4. 

Response of the model to a single stimulus. One stimulus is shown to the model for 2 sec, followed by a 4-sec delay period before resetting. The vertical axis is the normalized mean neuronal activity (i.e., normalized firing rate) in the different modules.

DMS Task

We ran simulations of the DMS condition using both our extended visual LSNM and the original LSNM (Ulloa & Horwitz, 2016; Tagamets & Horwitz, 1998). The simulated neuronal activities of the extended LSNM and the original LSNM are shown in Figure 5. As shown in Table 1, the accuracy of simulations run using the extended model is as high as the simulations run using the original model.

Figure 5. 

(A) Neuronal activities of the excitatory neurons in the different modules during one trial of the DMS task simulated using the extended model. (B) The excitatory neuronal activities of the different modules during one trial of DMS task simulated using the original model. In the simulations with the new model, the IT module showed activity during the delay period, which does not occur for the original model.

Figure 5. 

(A) Neuronal activities of the excitatory neurons in the different modules during one trial of the DMS task simulated using the extended model. (B) The excitatory neuronal activities of the different modules during one trial of DMS task simulated using the original model. In the simulations with the new model, the IT module showed activity during the delay period, which does not occur for the original model.

The new model also improved the performance of the IT area compared with the original model (see Figure 5). The module representing the IT area showed higher-than-baseline mean activation level during the delay periods in the DMS task, which eliminated the discrepancy between previous simulation and experimental results. As pointed out in the Introduction, the neurons of the IT module of the original LSNM of Tagamets and Horwitz (1998) did not show increased activity during the delay period of a DMS trial, in contradiction with nonhuman electrophysiological findings (Fuster et al., 1982). To address this small yet important disagreement, we added a feedback connection from the D2 module in pFC to IT cortex. Figure 5A shows the mean activities of selected modules in a simple DMS task from which the delay activation of the IT area can be seen.

When looking into individual behaviors of IT neurons of the extended model, we noticed that these neurons exhibited selectivity to different stimuli (Figure 6). Among activated IT neurons, most responded to all stimuli with or without delay activity, but neurons with selective activity can be observed in each trial. The data for the multiple-item holding Sternberg's task also displayed similar behaviors of stimulus selectivity. We did not observe similar behaviors in pFC. This difference between the model's IT cortex and pFC is consistent with experimental results (Miller et al., 1996) and shows that pFC is mostly involved in working memory and decision-making, thus supporting the idea that pFC neurons have little or no contribution in coding complex visual features.

Figure 6. 

Different types of excitatory neuronal activity and selectivity behaviors of simulated IT neurons during one DMS trial. Each of the gray stripes represents the presentation of one stimulus, and the white stripes between them represent the delay periods. We observed that the simulated neurons in the IT module exhibited several different activity patterns and selectivity to different stimuli. Most of activated IT neurons responded to both stimuli with or without delay activity (top), but in each trial, we observed neurons that responded only to the first or second stimulus (center and bottom); neurons with delay activity are shown at the bottom.

Figure 6. 

Different types of excitatory neuronal activity and selectivity behaviors of simulated IT neurons during one DMS trial. Each of the gray stripes represents the presentation of one stimulus, and the white stripes between them represent the delay periods. We observed that the simulated neurons in the IT module exhibited several different activity patterns and selectivity to different stimuli. Most of activated IT neurons responded to both stimuli with or without delay activity (top), but in each trial, we observed neurons that responded only to the first or second stimulus (center and bottom); neurons with delay activity are shown at the bottom.

DMS Task with Distractors and the “ABBA” Task

The model performed the DMS task with distractors and the “ABBA” task at a slightly lower accuracy rate (although still well above chance) than the simple DMS task (see Table 2). Figure 7A and B show the simulated neuronal activities of the different modules for the distractor task and the ABBA task, respectively. All modules (except the response module R) show increases in activation to all stimuli, but importantly, the response units (R) display activity greater than background noise only to the stimulus that matches the target. During the simulations shown in Figure 7A and B, four separate groups of entorhinal neuronal units responded respectively to the four items and passed the information to pFC for storage and comparison. The target (first stimulus) and intervening distractors are all stored in working memory modules (D1 and D2) but in separate groups of neuronal units during the simulation. The target is stored with high attention, whereas the intervening distractors are stored with lower attention. Consequently, the storage of distractors is weaker than the storage of targets; that is, fewer simulated neuronal units in their D1 and D2 modules showed persistent activity throughout the trial.

Table 2. 

Performances of 10 Simulated Participants during Four Tasks (DMS Task, DMS Task with Distractors, ABBA Task, and Sternberg's Recognition Task)

Participant DMS DMS with Distractors ABBA Sternberg 
S1 92.3% 90.5% 88.7% 90.3% 
S2 81.0% 80.5% 80.7% 78.7% 
S3 92.0% 92.0% 89.7% 87.0% 
S4 86.5% 84.0% 82.3% 82.0% 
S5 88.5% 89.0% 87.0% 87.7% 
S6 77.5% 75.5% 75.7% 73.0% 
S7 81.0% 79.5% 79.0% 78.7% 
S8 73.0% 69.5% 68.0% 69.3% 
S9 79.0% 79.5% 79.0% 78.0% 
S10 84.0% 81.0% 81.7% 79.7% 
Mean 84.5% 82.1% 81.2% 80.4% 
SD 6.02% 6.63% 6.15% 6.22% 
Participant DMS DMS with Distractors ABBA Sternberg 
S1 92.3% 90.5% 88.7% 90.3% 
S2 81.0% 80.5% 80.7% 78.7% 
S3 92.0% 92.0% 89.7% 87.0% 
S4 86.5% 84.0% 82.3% 82.0% 
S5 88.5% 89.0% 87.0% 87.7% 
S6 77.5% 75.5% 75.7% 73.0% 
S7 81.0% 79.5% 79.0% 78.7% 
S8 73.0% 69.5% 68.0% 69.3% 
S9 79.0% 79.5% 79.0% 78.0% 
S10 84.0% 81.0% 81.7% 79.7% 
Mean 84.5% 82.1% 81.2% 80.4% 
SD 6.02% 6.63% 6.15% 6.22% 

Performances are measured by counting the number of neuronal units in the decision-making module (R) firing above a certain threshold during the response period.

Figure 7. 

(A) Neuronal activities for DMS task with two intervening distractors. (B) The neuronal activities for the “ABBA” task. The two distractors were held in pFC with low attention (the persistent activities in D1 and D2 modules). The response module in both tasks properly avoided the distractors and responded when the probe was a match of the first stimulus.

Figure 7. 

(A) Neuronal activities for DMS task with two intervening distractors. (B) The neuronal activities for the “ABBA” task. The two distractors were held in pFC with low attention (the persistent activities in D1 and D2 modules). The response module in both tasks properly avoided the distractors and responded when the probe was a match of the first stimulus.

Sternberg's Recognition Task

Figure 8 shows the simulated neuronal activity of each module in three trials of Sternberg's recognition task, in which the model successfully responded to two matched cases and rejected one nonmatched case. Note that, in the first trial, the probe stimulus matches the first of the three stimuli being held in STM, whereas in the third trial, the probe matches the second presented stimulus. In both trials, the response units show increased activity, unlike the second trial where none of the stimuli match the probe. The average accuracy rates of 10 simulated participants can be found in Table 2.

Figure 8. 

Neuronal activities for Sternberg's task. Three trials of Sternberg's recognition task are shown. In each trial, the first three stimuli are the targets that the model needs to hold in working memory. In the first and last trials, the probe (last stimulus) was a match of one of the targets (the first and second, respectively), and the response module R made proper responses. In the second trial shown, the probe was not a match of any of the targets, and R did not respond to it.

Figure 8. 

Neuronal activities for Sternberg's task. Three trials of Sternberg's recognition task are shown. In each trial, the first three stimuli are the targets that the model needs to hold in working memory. In the first and last trials, the probe (last stimulus) was a match of one of the targets (the first and second, respectively), and the response module R made proper responses. In the second trial shown, the probe was not a match of any of the targets, and R did not respond to it.

In the simulated results of the Sternberg's recognition task, we noticed a significant primacy/recency effect. When the test stimulus is a match with the first or last item of the three items remembered, the model has a greater chance to make a correct response than when the test stimulus is a match with the second item (see Figure 9 and Table 3). The recency effect is stronger than the primacy effect. We will discuss our tentative explanation of this finding in the Discussion.

Figure 9. 

Mean accuracy (percent correct) for each participant for each position of the stimulus matching the probe stimulus of the Sternberg task. Blue bars correspond to the case when the matching stimulus was first, gray when it was second, and red when it was third. Nine of the 10 participants had more errors for the second position than for the other two, and nine participants had their highest accuracy for the third position (recency effect).

Figure 9. 

Mean accuracy (percent correct) for each participant for each position of the stimulus matching the probe stimulus of the Sternberg task. Blue bars correspond to the case when the matching stimulus was first, gray when it was second, and red when it was third. Nine of the 10 participants had more errors for the second position than for the other two, and nine participants had their highest accuracy for the third position (recency effect).

Table 3. 

Performance of 10 Simulated Participants during Sternberg's Task When the Probe Is a Match of the First, Second, and Third Targets, Respectively

 Target 1 Target 2 Target 3 
S1 91% 87% 93% 
S2 81% 76% 79% 
S3 86% 85% 90% 
S4 81% 80% 85% 
S5 88% 87% 88% 
S6 72% 70% 77% 
S7 78% 79% 79% 
S8 70% 66% 72% 
S9 78% 77% 79% 
S10 79% 78% 82% 
Mean 80% 79% 82% 
SD 6.28% 6.53% 6.17% 
 Target 1 Target 2 Target 3 
S1 91% 87% 93% 
S2 81% 76% 79% 
S3 86% 85% 90% 
S4 81% 80% 85% 
S5 88% 87% 88% 
S6 72% 70% 77% 
S7 78% 79% 79% 
S8 70% 66% 72% 
S9 78% 77% 79% 
S10 79% 78% 82% 
Mean 80% 79% 82% 
SD 6.28% 6.53% 6.17% 

In simulating the holding of multiple items in STM, one interesting finding related to the feedback from pFC to IT cortex is that many simulated neurons in both pFC and IT cortex displayed a progressive increase in activity level across the DMS trial with multiple distractors and the Sternberg task (Figure 10). This type of behavior could not be observed once we removed the feedback from pFC to IT cortex. We found neurons with climbing activity in both pFC and IT cortex, whereas monkey physiological studies have reported such activity only in pFC (Miller et al., 1996).

Figure 10. 

Simulated neurons with climbing activities during one trial of Sternberg's task. (A) Activity of one neuron found in pFC module. (B) Activity of one neuron found in the IT module. Each of the gray stripes indicates the presentation of one stimulus, and the white stripes between the gray are the delay periods.

Figure 10. 

Simulated neurons with climbing activities during one trial of Sternberg's task. (A) Activity of one neuron found in pFC module. (B) Activity of one neuron found in the IT module. Each of the gray stripes indicates the presentation of one stimulus, and the white stripes between the gray are the delay periods.

Simulated fMRI BOLD Signal

As discussed in the Methods section, we implemented an experiment that consisted of alternative blocks of DMS trials and control trials (passive viewing of degraded shapes) and then calculated the integrated synaptic activity and fMRI BOLD time series for select ROIs. Figure 11A and B show the integrated synaptic activity and fMRI BOLD signal, respectively, for ROIs during three blocks of DMS task (gray) and three blocks of the control task (white). Each block consists of three trials. We can conclude from the figures that the modules of higher order show more signal change between the DMS and control tasks. Early visual cortex V1/V2 did not show much change between DMS trials and control trials, but higher-order modules such as pFC and EC modules displayed much larger changes.

Figure 11. 

A simulated experiment with three blocks of alternative DMS trials (gray stripes) and control trials (white stripes) was implemented. Each block consisted of three trials. (A) The integrated synaptic activity of different modules. (B) The simulated fMRI BOLD signal of the different modules. Modules of higher order have larger difference in activity between the DMS and control tasks. (C) A comparison of the simulated synaptic activity and the fMRI BOLD signal of the V4 module during the simulated experiment for one task and one control block.

Figure 11. 

A simulated experiment with three blocks of alternative DMS trials (gray stripes) and control trials (white stripes) was implemented. Each block consisted of three trials. (A) The integrated synaptic activity of different modules. (B) The simulated fMRI BOLD signal of the different modules. Modules of higher order have larger difference in activity between the DMS and control tasks. (C) A comparison of the simulated synaptic activity and the fMRI BOLD signal of the V4 module during the simulated experiment for one task and one control block.

Figure 11C shows a comparison, using signals from V4 module as an example, between the integrated synaptic activity and fMRI BOLD signal. From the figure, we see that, using such a block design, the simulated BOLD signal cannot isolate the detailed response profile for each stimulus, as is well known to the experimental research community.

We also performed a simulated event-related experiment by extending the delay period to 20 sec to show a more complete response curve in BOLD signal for each incoming stimulus. Figure 12 shows the simulated fMRI BOLD signal for an event-related design that consists of a DMS trial, a DMS trial with two intervening distractors, and a Sternberg's task trial. The signal differences between these tasks and the control task (passive viewing of four stimuli separating each of the aforementioned tasks) were calculated and are shown in Table 4. By employing such a design, we could examine the effect of working memory load (by comparing the DMS task with the Sternberg task) and the effect of attention (by comparing the Sternberg's task with the DMS task with distractors). The V1/V2 module (the yellow trace in Figure 12) did not show statistically significant differences in terms of mean fMRI BOLD signals between the DMS tasks and the control tasks. We observed that the EC (black trace) displayed greater activity, compared with the control task, during encoding (stimuli presentations) and retrieval (probe presentations) but not during maintenance (delay periods), which is consistent with experimental findings (Schon, Quiroz, Hasselmo, & Stern, 2009). V4, IT cortex, and pFC showed increased activity during the delay periods, which indicated their roles in working memory maintenance. The activity of pFC increases when multiple items are stored, which agrees with the experimental evidence for both visual working memory (Cairo, Liddle, Woodward, & Ngan, 2004; Druzgal & D'Esposito, 2003; Rypma, Berger, & D'Esposito, 2002; Rypma & D'Esposito, 1999) and verbal working memory (Veltman, Rombouts, & Dolan, 2003). We also noticed the effect of working memory load for V4 and IT during encoding and maintenance (stimuli presentations and delay periods), but not during the retrieval phase (probe presentation).

Figure 12. 

The simulated fMRI BOLD signals of different modules of an event-related experiment. Each gray bar represents the presentation of a stimulus. The experiment consists of one DMS trial (first two gray bars), one DMS trial with two distractors (the middle four gray bars), and one trial of Sternberg's task (the last four gray bars). Passive viewing of four stimuli separates each of these task trials. The V4 (orange), IT cortex (blue), EC (black), and pFC (red) modules showed higher fMRI BOLD signals when the working memory load increased (DMS vs. Sternberg's task).

Figure 12. 

The simulated fMRI BOLD signals of different modules of an event-related experiment. Each gray bar represents the presentation of a stimulus. The experiment consists of one DMS trial (first two gray bars), one DMS trial with two distractors (the middle four gray bars), and one trial of Sternberg's task (the last four gray bars). Passive viewing of four stimuli separates each of these task trials. The V4 (orange), IT cortex (blue), EC (black), and pFC (red) modules showed higher fMRI BOLD signals when the working memory load increased (DMS vs. Sternberg's task).

Table 4. 

The Mean Signal Change (in Percentage) of Specific Task from Control Task (Passive Viewing)

 Integrated Synaptic Activity fMRI 
DMS DMS with Distractors Sternberg's Task DMS DMS with Distractors Sternberg's Task 
V1 6.533* 5.318* 8.702* 2.739 1.841 3.028 
V4 25.786* 27.861* 29.250* 23.034* 25.548* 26.947* 
IT 13.318* 16.470* 18.852* 11.750* 14.824* 15.773* 
EC 8.161* 8.803* 10.182* 5.709* 8.871* 9.098* 
pFC 25.083* 28.915* 32.594* 20.623* 23.647* 26.992* 
 Integrated Synaptic Activity fMRI 
DMS DMS with Distractors Sternberg's Task DMS DMS with Distractors Sternberg's Task 
V1 6.533* 5.318* 8.702* 2.739 1.841 3.028 
V4 25.786* 27.861* 29.250* 23.034* 25.548* 26.947* 
IT 13.318* 16.470* 18.852* 11.750* 14.824* 15.773* 
EC 8.161* 8.803* 10.182* 5.709* 8.871* 9.098* 
pFC 25.083* 28.915* 32.594* 20.623* 23.647* 26.992* 

Both integrated synaptic activity and fMRI data of different brain regions are shown. Paired t test was used for the signal difference.

*

p < .05.

DISCUSSION

We have presented a simulation study of several STM tasks using an extended version of a previously constructed LSNM. A module representing the EC to serve as a gating module was added to process multiple items. We successfully implemented multiple STM tasks using this extended model and produced neuronal patterns in visual cortex, IT cortex, EC, and pFC that match experimental findings. These STM tasks can include distractor stimuli or can require that multiple items be retained in mind during a delay period.

Fuster and Jervey (1982) first revealed in primate single-unit recording studies that IT neurons exhibit sustained, increased activity during the short delay of a DMS task. A number of later studies have also supported the notion that IT cortex is important for the maintenance of visual object information (Ranganath & D'Esposito, 2005; Petrides, 2000; Horel et al., 1987). The neuronal behaviors in IT across short delays in DMS trials were indicated to be relevant to object-selective activity and associative learning (Erickson & Desimone, 1999; Miller et al., 1993; Miyashita, 1988). The extended version of our LSNM, compared with the original version, explicitly implements this critical role for IT, which will be important for extending our model to incorporate a long-term memory component.

Fuster and colleagues first reported the presence of pFC neurons with climbing activity across delay periods in DMS trials without distractors (Quintana & Fuster, 1992). Similar behavior was also observed in DMS trials with distractors (Miller et al., 1996). Fuster and colleagues interpreted the climbing activity as expectation. Our model does not have expectation built-in. Rather, our simulations suggest that the climbing activity is related to feedback from pFC to IT cortex; that is, a recurrent loop is formed with feedback from pFC to IT cortex so that the information stored in pFC can strengthen itself through the loop, and it represents the working memory distributed in the network. These two notions of the neural mechanism for the observed climbing behavior may in fact complement one another.

We modeled working memory using the D1–D2 microcircuit as a fixed state. Meanwhile, the recurrent connectivity between pFC and IT cortex in our model enables a network-dependent mechanism of working memory represented by the “climbing neurons” we observed. There is experimental evidence for both views of working memory. Persistent activity of neurons in pFC was observed during the delay period by Funahashi et al. (1990), whereas some neuronal activity in pFC declined and was reactivated during the delay period (Barak, Tsodyks, & Romo, 2010; Rainer & Miller, 2002), indicating the existence of a dynamic mechanism for working memory.

In the simulations of DMS with distractors and the “ABBA” task, we assumed that the intervening distractors are also stored in pFC. Because of the low attention level applied, the storage of distractors is weaker than the storage of targets. Experimental studies in visual search and incidental learning support our assumption that the distractors are stored in working memory (Goolsby, Shapiro, & Raymond, 2009; Williams, Henderson, & Zacks, 2005). We used separate working memory modules to handle distractors in the DMS task based on the fact that the attention paid to the distractors is lower than the target and the target has a special status in working memory that is not shared by the distractors (Peters, Goebel, & Roelfsema, 2009). The structural network of multiple working memory modules was inspired by a similar scheme proposed by Ulloa et al. in an auditory model that dealt with long-duration tonal patterns (Ulloa et al., 2008). The capacity limit of working memory is implemented by limiting the number of memory pools (we used three, but the number is arbitrary). Once the memory pools are all filled, further items will not be stored, and the corresponding BOLD signal will reach a plateau. This was instantiated in our model for simplicity, but future research could aim toward determining whether this assumption is warranted.

Whereas the classic view is that working memory has a limited number of slots (Cowan, 2001), some recent experimental and modeling studies propose working memory as a continuous resource that is distributed among all remembered items (Keshvari, van den Berg, & Ma, 2013; Fougnie, Suchow, & Alvarez, 2012; Bays & Husain, 2008; for a review, see Ma, Husain, & Bays, 2014). According to this view, the precision of memory, which decreases as more items are remembered, is the key metric of working memory limits instead of the quantity of memory items. In our model, the working memory representations of different items are stored in nonoverlapping pFC modules, which, in the future, could be integrated into one continuous module.

Although we did not explicitly set out to incorporate a primacy and recency effect in our model, nonetheless, we observed such effects in our simulation results. In our model, the observed primacy effect in the Sternberg task was a result of decayed attention. In our simulated experimental design, the attention applied to the prefrontal area decays with time, and higher attention helps the working memory network encoding for new items. The neural basis underlying the experimentally observed recency effect has been debated (Baddeley & Hitch, 1993). On the basis of our simulation, we suggest that the gating mechanism, specifically competitive inhibition and the inhibitory feedback from pFC to EC, may contribute to the recency effect. The inhibitory feedback from pFC to entorhinal area reduces the competition level among gating neurons; thus, later incoming stimuli have less inhibition and stronger representations in working memory. Previous experiments have shown that these effects are sensitive to the duration of the delay periods (Wright, 1999), which we have not observed in our simulation study, possibly because of a lack of a “forgetting” mechanism in the current version of the model.

Because working memory is such an important cognitive process, many research groups have developed models of this process. They range from purely cognitive models (Baddeley, 1992) to computational models of varying levels of complexity (e.g., Rolls, Dempere-Marco, & Deco, 2013; Ashby, Ell, Valentin, & Casale, 2005; Amit, Fusi, & Yakovlev, 1997; Dehaene & Changeux, 1989; for reviews, see Barak & Tsodyks, 2014; Maex & Steuber, 2009; Durstewitz, Seamans, & Sejnowski, 2000). Many computational models aimed to account for both behavioral and neural activity observed in monkey electrophysiological studies during the delay portion of a delayed response task. For instance, one approach, exemplified by Amit et al. (1997), employed recurrent excitatory connections in a cell assembly to maintain stable activity patterns (i.e., attractors). In initial studies, Hebb-like learning methods were employed to generate synaptic weights that reinforced the connections between specific neurons. These attractor models initially dealt with maintaining in STM one or more previously learned images. More recent work has extended these models so that novel images can also be handled (e.g., Amit, Yakovlev, & Hochstein, 2013). Although a number of modeling efforts addressing working memory have focused on pFC, a substantial number have also argued that the BG play an important role as well (e.g., Ashby et al., 2005; Monchi & Taylor, 1999). The modeling framework proposed by Ashby et al. (2005) is of particular interest because, like the model we presented in this article, it also provides a distributed neurocomputational model that incorporates multiple, interacting brain regions and aims to account for both neurophysiological data and behavioral data. Furthermore, the authors argue that this approach, like ours, also can account for human neuroimaging data (Ashby & Valentin, 2007).

The BG and the thalamus have been implicated in working memory function. Lesions of caudate and medial dorsal nuclei of the thalamus can severely impair working memory capacity (Kubat-Silman, Dagenbach, & Absher, 2002). However, as pointed out by Ashby et al. (2005), lesions of the caudate and medial dorsal nuclei of the thalamus can impair but will not abolish working memory, and this has been found experimentally (e.g., Janahashi et al., 2002; Gabrieli, Singh, Stebbins, & Goetz, 1996). It is worth noting that the examples used in this article deal with a rather limited working memory capacity (i.e., no more than three objects).

The interactions between pFC and the BG and the thalamus have also been interpreted as a gating mechanism (O'Reilly & Frank, 2006; Braver & Cohen, 2000; Cohen, Braver, & O'Reilly, 1996). For example, the O'Reilly-Frank working memory model incorporates a pFC that controls both itself and other brain areas in a task-dependent manner. It does this by employing learning mechanisms that involve a number of subcortical structures including the BG that act as a gating mechanism for updating working memory. However, the EC is considered to be directly involved in the visual ventral (object) processing pathway and declarative memory encoding (Preston & Eichenbaum, 2013).

In the current article, we chose parameters for the added components of the model so as to provide a reasonable match to the electrophysiological data, although we did not employ explicit model-fitting to any particular data set. As pointed out by Ashby et al. (2005), there is much variability between cells in monkey electrophysiological data, which may preclude quantitative data fitting. Moreover, another reason that explicit data fitting was not employed was that there are numerous data sets (behavioral performance, electrophysiological data in multiple brain regions, fMRI activation and connectivity data, MEG/EEG data) that we want our model to account for. These data have different featural and temporal characteristics. It is not clear to us how in principle one should go about fitting all these data simultaneously. In fact, as far as the neuroimaging data are concerned, other researchers (e.g., Friston et al., in press) have only begun to develop a systematic approach to this problem.

Some caveats of our work include that the attention level and the top–down task control we used in the model are not realistically modeled; we hypothesized that the EC was responsible for a gating process, which needs to be confirmed by experiments, and the locations we chose for prefrontal nodes (D1, D2, FS, R) in TVB are somewhat arbitrary. We plan to address these caveats in future work and extend the model to incorporate long-term memory and implement related cognitive tasks (e.g., paired associate task).

In summary, we have performed several STM tasks using one LSNM and studied various neuronal behaviors in IT cortex and pFC. We modeled working memory with local microcircuits (D1, D2) and a large-scale recurrent network (pFC, IT cortex), which produced neuronal behaviors that matched experimental findings. For generating a brain-like environment, we embedded the model into TVB framework. The model in the future can be extended to incorporate more brain regions and functions, such as long-term memory. Our results indicate that computational modeling can be a powerful tool for interpreting human and nonhuman primate neuroimaging data.

Acknowledgments

The authors thank Dr. Paul Corbitt for useful discussions and John Gilbert with help with incorporating our model into the TVB framework. They also thank Drs. Fatima Husain and Daniel Butts for comments on the article. The research was supported by the NIH/NIDCD Intramural Research Program.

Reprint requests should be sent to Qin Liu, Brain Imaging & Modeling Section, NIDCD-NIH, Bethesda, MD 20892, or via e-mail: qin.liu@nih.gov.

REFERENCES

Amit
,
D. J.
,
Fusi
,
S.
, &
Yakovlev
,
V.
(
1997
).
Paradigmatic working memory (attractor) cell in IT cortex
.
Neural Computation
,
9
,
1071
1092
.
Amit
,
Y.
,
Yakovlev
,
V.
, &
Hochstein
,
S.
(
2013
).
Modeling behavior in different delay match to sample tasks in one simple network
.
Frontiers in Human Neuroscience
,
7
,
408
.
Ashby
,
F. G.
,
Ell
,
S. W.
,
Valentin
,
V. V.
, &
Casale
,
M. B.
(
2005
).
FROST: A distributed neurocomputational model of working memory maintenance
.
Journal of Cognitive Neuroscience
,
17
,
1728
1743
.
Ashby
,
F. G.
, &
Valentin
,
V. V.
(
2007
).
Computational cognitive neuroscience: Building and testing biologically plausible computational models of neuroscience, neuroimaging, and behavioral data
. In
M. J.
Wenger
&
C.
Schuster
(Eds.),
Statistical and process models for cognitive neuroscience and aging
(pp.
15
58
).
Mahwah, NJ
:
Erlbaum
.
Baddeley
,
A.
(
1992
).
Working memory: The interface between memory and cognition
.
Journal of Cognitive Neuroscience
,
4
,
281
288
.
Baddeley
,
A. D.
, &
Hitch
,
G.
(
1993
).
The recency effect: Implicit learning with explicit retrieval?
Memory & Cognition
,
21
,
146
155
.
Barak
,
O.
, &
Tsodyks
,
M.
(
2014
).
Working models of working memory
.
Current Opinion in Neurobiology
,
25
,
20
24
.
Barak
,
O.
,
Tsodyks
,
M.
, &
Romo
,
R.
(
2010
).
Neuronal population coding of parametric working memory
.
Journal of Neuroscience
,
30
,
9424
9430
.
Bays
,
P. M.
, &
Husain
,
M.
(
2008
).
Dynamic shifts of limited working memory resources in human vision
.
Science
,
321
,
851
854
.
Braver
,
T. S.
, &
Cohen
,
J. D.
(
2000
).
On the control of control: The role of dopamine in regulating prefrontal function and working memory
. In
S.
Monsell
&
J.
Driver
(Eds.),
Control of cognitive processes: Attention and performance
(
Vol. XVIII
, pp.
713
737
).
Cambridge, MA
:
MIT Press
.
Cairo
,
T. A.
,
Liddle
,
P. F.
,
Woodward
,
T. S.
, &
Ngan
,
E. T.
(
2004
).
The influence of working memory load on phase specific patterns of cortical activity
.
Brain Research. Cognitive Brain Research
,
21
,
377
387
.
Chaudhuri
,
R.
,
Knoblauch
,
K.
,
Gariel
,
M. A.
,
Kennedy
,
H.
, &
Wang
,
X. J.
(
2015
).
A large-scale circuit mechanism for hierarchical dynamical processing in the primate cortex
.
Neuron
,
88
,
419
431
.
Cohen
,
J. D.
,
Braver
,
T. S.
, &
O'Reilly
,
R. C.
(
1996
).
A computational approach to prefrontal cortex, cognitive control and schizophrenia: Recent developments and current challenges
.
Philosophical Transactions of the Royal Society of London, Series B, Biological Sciences
,
351
,
1515
1527
.
Corchs
,
S.
, &
Deco
,
G.
(
2002
).
Large-scale neural model for visual attention: Integration of experimental single-cell and fMRI data
.
Cerebral Cortex
,
12
,
339
348
.
Courtney
,
S. M.
,
Petit
,
L.
,
Maisog
,
J. M.
,
Ungerleider
,
L. G.
, &
Haxby
,
J. V.
(
1998
).
An area specialized for spatial working memory in human frontal cortex
.
Science
,
279
,
1347
1351
.
Cowan
,
N.
(
2001
).
The magic number 4 in short-term memory: A reconsideration of mental storage capacity
.
Behavioral and Brain Sciences
,
24
,
87
114
.
Dehaene
,
S.
, &
Changeux
,
J. P.
(
1989
).
A simple model of prefrontal cortex function in delayed-response tasks
.
Journal of Cognitive Neuroscience
,
1
,
244
261
.
Desimone
,
R.
, &
Schein
,
S. J.
(
1987
).
Visual properties of neurons in area V4 of the macaque: Sensitivity to stimulus form
.
Journal of Neurophysiology
,
57
,
835
868
.
D'Esposito
,
M.
,
Detre
,
J. A.
,
Alsop
,
D. C.
,
Shin
,
R. K.
,
Atlas
,
S.
, &
Grossman
,
M.
(
1995
).
The neural basis of the central executive system of working memory
.
Nature
,
378
,
279
281
.
Druzgal
,
T. J.
, &
D'Esposito
,
M.
(
2003
).
Dissecting contributions of prefrontal cortex and fusiform face area to face working memory
.
Journal of Cognitive Neuroscience
,
15
,
771
784
.
Durstewitz
,
D.
,
Seamans
,
J. K.
, &
Sejnowski
,
T. J.
(
2000
).
Neurocomputational models of working memory
.
Nature Neuroscience
,
3(Suppl.)
,
1184
1191
.
Erickson
,
C. A.
, &
Desimone
,
R.
(
1999
).
Responses of macaque perirhinal neurons during and after visual stimulus association learning
.
Journal of Neuroscience
,
19
,
10404
10416
.
Fougnie
,
D.
,
Suchow
,
J. W.
, &
Alvarez
,
G. A.
(
2012
).
Variability in the quality of visual working memory
.
Nature Communications
,
3
,
1229
.
Friston
,
K. J.
,
Preller
,
K. H.
,
Mathys
,
C.
,
Cagnan
,
H.
,
Heinzle
,
J.
,
Razi
,
A.
, et al
(
in press
).
Dynamic causal modelling revisited
.
Neuroimage
.
doi:10.1016/j.neuroimage.2017.02.045
.
Funahashi
,
S.
,
Bruce
,
C. J.
, &
Goldman-Rakic
,
P. S.
(
1989
).
Mnemonic coding of visual space in the monkey's dorsolateral prefrontal cortex
.
Journal of Neurophysiology
,
61
,
331
349
.
Funahashi
,
S.
,
Bruce
,
C. J.
, &
Goldman-Rakic
,
P. S.
(
1990
).
Visuospatial coding in primate prefrontal neurons revealed by oculomotor paradigms
.
Journal of Neurophysiology
,
63
,
814
831
.
Funahashi
,
S.
,
Inoue
,
M.
, &
Kubota
,
K.
(
1993
).
Delay-related activity in the primate prefrontal cortex during sequential reaching tasks with delay
.
Neuroscience Research
,
18
,
171
175
.
Fuster
,
J.
,
Bauer
,
R.
, &
Jervey
,
J.
(
1982
).
Cellular discharge in the dorsolateral prefrontal cortex of the monkey in cognitive tasks
.
Experimental Neurology
,
77
,
679
694
.
Fuster
,
J. M.
, &
Alexander
,
G. E.
(
1971
).
Neuron activity related to short-term memory
.
Science
,
173
,
652
654
.
Fuster
,
J. M.
, &
Alexander
,
G. E.
(
1973
).
Firing changes in cells of the nucleus medialis dorsalis associated with delayed response behavior
.
Brain Research
,
61
,
79
91
.
Fuster
,
J. M.
, &
Jervey
,
J. P.
(
1982
).
Neuronal firing in the inferotemporal cortex of the monkey in a visual memory task
.
Journal of Neuroscience
,
2
,
361
375
.
Gabrieli
,
J. D.
,
Singh
,
J.
,
Stebbins
,
G. T.
, &
Goetz
,
C. G.
(
1996
).
Reduced working memory span in Parkinson's disease: Evidence of the role of a frontostriatal system in working and strategic memory
.
Neuropsychology
,
10
,
322
332
.
Gallant
,
J. L.
,
Braun
,
J.
, &
Van Essen
,
D. C.
(
1993
).
Selectivity for polar, hyperbolic, and Cartesian gratings in macaque visual cortex
.
Science
,
259
,
100
103
.
Garagnani
,
M.
,
Wennekers
,
T.
, &
Pulvermuller
,
F.
(
2008
).
A neuroanatomically grounded Hebbian-learning model of attention-language interactions in the human brain
.
European Journal of Neuroscience
,
27
,
492
513
.
Gisiger
,
T.
, &
Kerszberg
,
M.
(
2006
).
A model for integrating elementary neural functions into delayed-response behavior
.
PLoS Computational Biology
,
2
,
e25
.
Goolsby
,
B. A.
,
Shapiro
,
K. L.
, &
Raymond
,
J. E.
(
2009
).
Distractor devaluation requires visual working memory
.
Psychonomic Bulletin & Review
,
16
,
133
138
.
Hagmann
,
P.
,
Cammoun
,
L.
,
Gigandet
,
X.
,
Meuli
,
R.
,
Honey
,
C. J.
,
Wedeen
,
V. J.
, et al
(
2008
).
Mapping the structural core of human cerebral cortex
.
PLoS Biology
,
6
,
e159
.
Haxby
,
J.
,
Ungerleider
,
L.
,
Horwitz
,
B.
,
Rapoport
,
S.
, &
Grady
,
C.
(
1995
).
Hemispheric differences in neural systems for face working memory: A PET-rCBF study
.
Human Brain Mapping
,
3
,
68
82
.
Haxby
,
J. V.
,
Grady
,
C. L.
,
Horwitz
,
B.
,
Ungerleider
,
L. G.
,
Mishkin
,
M.
,
Carson
,
R. E.
, et al
(
1991
).
Dissociation of object and spatial visual processing pathways in human extrastriate cortex
.
Proceedings of the National Academy of Sciences, U.S.A.
,
88
,
1621
1625
.
Horel
,
J. A.
,
Pytko-Joiner
,
D. E.
,
Voytko
,
M. L.
, &
Salsbury
,
K.
(
1987
).
The performance of visual tasks while segments of the inferotemporal cortex are suppressed by cold
.
Behavioural Brain Research
,
23
,
29
42
.
Horwitz
,
B.
, &
Banerjee
,
A.
(
2012
).
A role for neural modeling in the study of brain disorders
.
Frontiers in Systems Neuroscience
,
6
,
57
.
Horwitz
,
B.
, &
Tagamets
,
M. A.
(
1999
).
Predicting human functional maps with neural net modeling
.
Human Brain Mapping
,
8
,
137
142
.
Hubel
,
D. H.
,
Wiesel
,
T. N.
, &
Stryker
,
M. P.
(
1977
).
Orientation columns in macaque monkey visual cortex demonstrated by the 2-deoxyglucose autoradiographic technique
.
Nature
,
269
,
328
330
.
Husain
,
F. T.
,
Nandipati
,
G.
,
Braun
,
A. R.
,
Cohen
,
L. G.
,
Tagamets
,
M. A.
, &
Horwitz
,
B.
(
2002
).
Simulating transcranial magnetic stimulation during PET with a large-scale neural network model of the prefrontal cortex and the visual system
.
Neuroimage
,
15
,
58
73
.
Husain
,
F. T.
,
Tagamets
,
M. A.
,
Fromm
,
S. J.
,
Braun
,
A. R.
, &
Horwitz
,
B.
(
2004
).
Relating neuronal dynamics for auditory object processing to neuroimaging activity: A computational modeling and an fMRI study
.
Neuroimage
,
21
,
1701
1720
.
Janahashi
,
M.
,
Rowe
,
J.
,
Saleem
,
T.
,
Brown
,
R.
,
Limousin-Dowsey
,
P.
,
Rothwell
,
J.
, et al
(
2002
).
Striatal contribution to cognition: Working memory and executive function in Parkinson's disease before and after unilateral posteroventral pallidotomy
.
Journal of Cognitive Neuroscience
,
14
,
298
310
.
Keshvari
,
S.
,
van den Berg
,
R.
, &
Ma
,
W. J.
(
2013
).
No evidence for an item limit in change detection
.
PLoS Computational Biology
,
9
,
e1002927
.
Kim
,
J.
, &
Horwitz
,
B.
(
2009
).
How well does structural equation modeling reveal abnormal brain anatomical connections? An fMRI simulation study
.
Neuroimage
,
45
,
1190
1198
.
Kubat-Silman
,
A. K.
,
Dagenbach
,
D.
, &
Absher
,
J. R.
(
2002
).
Patterns of impaired verbal, spatial, and object working memory after thalamic lesions
.
Brain and Cognition
,
50
,
178
193
.
Kubota
,
K.
, &
Niki
,
H.
(
1971
).
Prefrontal cortical unit activity and delayed alternation performance in monkeys
.
Journal of Neurophysiology
,
34
,
337
347
.
Lech
,
R. K.
, &
Suchan
,
B.
(
2014
).
Involvement of the human medial temporal lobe in a visual discrimination task
.
Behavioural Brain Research
,
268
,
22
30
.
Levy
,
R.
, &
Goldman-Rakic
,
P. S.
(
1999
).
Association of storage and processing functions in the dorsolateral prefrontal cortex of the nonhuman primate
.
Journal of Neuroscience
,
19
,
5149
5158
.
Ma
,
W. J.
,
Husain
,
M.
, &
Bays
,
P. M.
(
2014
).
Changing concepts of working memory
.
Nature Neuroscience
,
17
,
347
356
.
Maex
,
R.
, &
Steuber
,
V.
(
2009
).
The first second: Models of short-term memory traces in the brain
.
Neural Networks
,
22
,
1105
1112
.
Miller
,
E. K.
,
Erickson
,
C. A.
, &
Desimone
,
R.
(
1996
).
Neural mechanisms of visual working memory in prefrontal cortex of the macaque
.
Journal of Neuroscience
,
16
,
5154
5167
.
Miller
,
E. K.
,
Gochin
,
P. M.
, &
Gross
,
C. G.
(
1993
).
Suppression of visual responses of neurons in inferior temporal cortex of the awake macaque by addition of a second stimulus
.
Brain Research
,
616
,
25
29
.
Miller
,
G. A.
(
1956
).
The magical number seven plus or minus two: Some limits on our capacity for processing information
.
Psychological Review
,
63
,
81
97
.
Mishkin
,
M.
, &
Ungerleider
,
L. G.
(
1982
).
Contribution of striate inputs to the visuospatial functions of parieto-preoccipital cortex in monkeys
.
Behavioural Brain Research
,
6
,
57
77
.
Miyashita
,
Y.
(
1988
).
Neuronal correlate of visual associative long-term memory in the primate temporal cortex
.
Nature
,
335
,
817
820
.
Monchi
,
O.
, &
Taylor
,
J. G.
(
1999
).
A hard wired model of coupled frontal working memories for various tasks
.
Information Sciences
,
113
,
221
243
.
Navawongse
,
R.
, &
Eichenbaum
,
H.
(
2013
).
Distinct pathways for rule-based retrieval and spatial mapping of memory representations in hippocampal neurons
.
Journal of Neuroscience
,
33
,
1002
1013
.
O'Reilly
,
R. C.
, &
Frank
,
M. J.
(
2006
).
Making working memory work: A computational model of learning in the prefrontal cortex and basal ganglia
.
Neural Computation
,
18
,
283
328
.
Peterhans
,
E.
, &
von der Heydt
,
R.
(
1993
).
Functional organization of area V2 in the alert macaque
.
European Journal of Neuroscience
,
5
,
509
524
.
Peters
,
J. C.
,
Goebel
,
R.
, &
Roelfsema
,
P. R.
(
2009
).
Remembered but unused: The accessory items in working memory that do not guide attention
.
Journal of Cognitive Neuroscience
,
21
,
1081
1091
.
Petrides
,
M.
(
2000
).
Dissociable roles of mid-dorsolateral prefrontal and anterior inferotemporal cortex in visual working memory
.
Journal of Neuroscience
,
20
,
7496
7503
.
Preston
,
A. R.
, &
Eichenbaum
,
H.
(
2013
).
Interplay of hippocampus and prefrontal cortex in memory
.
Current Biology
,
23
,
R764
R773
.
Quintana
,
J.
, &
Fuster
,
J. M.
(
1992
).
Mnemonic and predictive functions of cortical neurons in a memory task
.
NeuroReport
,
3
,
721
724
.
Rainer
,
G.
, &
Miller
,
E. K.
(
2002
).
Timecourse of object-related neural activity in the primate prefrontal cortex during a short-term memory task
.
European Journal of Neuroscience
,
15
,
1244
1254
.
Ranganath
,
C.
, &
D'Esposito
,
M.
(
2005
).
Directing the mind's eye: Prefrontal, inferior and medial temporal mechanisms for visual working memory
.
Current Opinion in Neurobiology
,
15
,
175
182
.
Riley
,
M. R.
, &
Constantinidis
,
C.
(
2015
).
Role of prefrontal persistent activity in working memory
.
Frontiers in Systems Neuroscience
,
9
,
181
.
Roe
,
A. W.
, &
Ts'o
,
D. Y.
(
1995
).
Visual topography in primate V2: Multiple representation across functional stripes
.
Journal of Neuroscience
,
15
,
3689
3715
.
Rolls
,
E. T.
,
Dempere-Marco
,
L.
, &
Deco
,
G.
(
2013
).
Holding multiple items in short term memory: A neural mechanism
.
PLoS One
,
8
,
e61078
.
Rypma
,
B.
,
Berger
,
J. S.
, &
D'Esposito
,
M.
(
2002
).
The influence of working-memory demand and subject performance on prefrontal cortical activity
.
Journal of Cognitive Neuroscience
,
14
,
721
731
.
Rypma
,
B.
, &
D'Esposito
,
M.
(
1999
).
The roles of prefrontal brain regions in components of working memory: Effects of memory load and individual differences
.
Proceedings of the National Academy of Sciences, U.S.A.
,
96
,
6558
6563
.
Saleem
,
K. S.
, &
Tanaka
,
K.
(
1996
).
Divergent projections from the anterior inferotemporal area TE to the perirhinal and entorhinal cortices in the macaque monkey
.
Journal of Neuroscience
,
16
,
4757
4775
.
Sanz Leon
,
P.
,
Knock
,
S. A.
,
Woodman
,
M. M.
,
Domide
,
L.
,
Mersmann
,
J.
,
McIntosh
,
A. R.
, et al
(
2013
).
The virtual brain: A simulator of primate brain network dynamics
.
Frontiers in Neuroinformatics
,
7
,
10
.
Schon
,
K.
,
Quiroz
,
Y. T.
,
Hasselmo
,
M. E.
, &
Stern
,
C. E.
(
2009
).
Greater working memory load results in greater medial temporal activity at retrieval
.
Cerebral Cortex
,
19
,
2561
2571
.
Schon
,
K.
,
Ross
,
R. S.
,
Hasselmo
,
M. E.
, &
Stern
,
C. E.
(
2013
).
Complementary roles of medial temporal lobes and mid-dorsolateral prefrontal cortex for working memory for novel and familiar trial-unique visual stimuli
.
European Journal of Neuroscience
,
37
,
668
678
.
Stephan
,
K. E.
,
Weiskopf
,
N.
,
Drysdale
,
P. M.
,
Robinson
,
P. A.
, &
Friston
,
K. J.
(
2007
).
Comparing hemodynamic models with DCM
.
Neuroimage
,
38
,
387
401
.
Sternberg
,
S.
(
1966
).
High-speed scanning in human memory
.
Science
,
153
,
652
654
.
Sternberg
,
S.
(
1969
).
Memory-scanning: Mental processes revealed by reaction-time experiments
.
American Scientist
,
57
,
421
457
.
Tagamets
,
M. A.
, &
Horwitz
,
B.
(
1998
).
Integrating electrophysiological and anatomical experimental data to create a large-scale model that simulates a delayed match-to-sample human brain imaging study
.
Cerebral Cortex
,
8
,
310
320
.
Talairach
,
J.
, &
Tournoux
,
P.
(
1988
).
Co-planar stereotaxic atlas of the human brain
.
New York
:
Thieme Medical Publishers, Inc.
Ulloa
,
A.
, &
Horwitz
,
B.
(
2016
).
Embedding task-based neural models into a connectome-based model of the cerebral cortex
.
Frontiers in Neuroinformatics
,
10
,
32
.
Ulloa
,
A.
,
Husain
,
F. T.
,
Kemeny
,
S.
,
Xu
,
J.
,
Braun
,
A. R.
, &
Horwitz
,
B.
(
2008
).
Neural mechanisms of auditory discrimination of long-duration tonal patterns: A neural modeling and fMRI study
.
Journal of Integrative Neuroscience
,
7
,
501
527
.
Veltman
,
D. J.
,
Rombouts
,
S. A.
, &
Dolan
,
R. J.
(
2003
).
Maintenance versus manipulation in verbal working memory revisited: An fMRI study
.
Neuroimage
,
18
,
247
256
.
Williams
,
C. C.
,
Henderson
,
J. M.
, &
Zacks
,
R. T.
(
2005
).
Incidental visual memory for targets and distractors in visual search
.
Perception & Psychophysics
,
67
,
816
827
.
Wilson
,
H. R.
, &
Cowan
,
J. D.
(
1972
).
Excitatory and inhibitory interactions in localized populations of model neurons
.
Biophysical Journal
,
12
,
1
24
.
Wright
,
A. A.
(
1999
).
Auditory list memory and interference processes in monkeys
.
Journal of Experimental Psychology: Animal Behavior Processes
,
25
,
284
296
.