Converging evidence from humans and nonhuman primates is obliging us to abandon conventional models in favor of a radically different, distributed-network paradigm of cortical memory. Central to the new paradigm is the concept of memory network or cognit—that is, a memory or an item of knowledge defined by a pattern of connections between neuron populations associated by experience. Cognits are hierarchically organized in terms of semantic abstraction and complexity. Complex cognits link neurons in noncontiguous cortical areas of prefrontal and posterior association cortex. Cognits overlap and interconnect profusely, even across hierarchical levels (heterarchically), whereby a neuron can be part of many memory networks and thus many memories or items of knowledge.
The history of cognitive neuroscience began in the 19th century with the controversy between phrenologists and experimentalists about the cerebral localization of functions and with Broca's (1861) publication of the disorder of language from frontal injury that bears his name (Young, 1990). Since that time, the field has been divided into two camps or schools of thought. In one are those who advocate that each complex cognitive function is localized in a separate part of the cerebral cortex, as Broca advocated with respect to articulated speech. In the other are those who maintain that complex cognitive functions are widely distributed in the cortex, as is the information they use. Until now, however, this second position has remained in the shadows for lack of empirical support, whereas modular views of cognition have thrived, largely inspired by the successes of reductionism in most all other fields of neuroscience. The cognitive neuroscience of memory has evolved on both sides of that conceptual divide.
Two lines of evidence traditionally support the localization of memory in the cortex, that is, the allocation of a given cortical area or an anatomical module to a given memory content: (1) discrete cortical lesions cause discrete memory deficits, and (2) electrical stimulation at certain locations, especially in cortex of association, can elicit vivid memories. Further, modular views of memory have been inferred from cortical sensory physiology. Sensory qualities are represented in discrete module-like areas of sensory cortex. From this evidence derives the unproven assumption that, beyond those sensory areas, perceptual memory is represented in modules of association cortex. At most, however, those lines of evidence or extrapolation indicate that some cortical areas are more related to one kind of memory than to others.
Nonetheless, modular concepts are ubiquitous in cognitive neuroscience. A functional module, as generally understood, is a continuous and circumscribed portion of cortex dedicated to one particular function and not others. In cortical physiology, certain anatomical configurations of neural elements (e.g., microscopic columns) have been identified as functional modules inasmuch as they contain geometrical arrangements of neurons specialized in a particular sensory or motor function. Arguably, even beyond primary sensory and motor cortices, certain circumscribed areas are functionally modular, in that they specialize in discrete physiological functions such as the detection of visual movement (area MT) or ocular motility (FEFs). A serious problem arises, however, when a cognitive function such as perception, memory, attention, language, or intelligence is ascribed to a discrete module of cortex as defined above. Modular models are all based on that definition of a module, which at least with regard to memory is theoretically and empirically inconsistent with the recent literature.
Network memory models, on the other hand, are not incompatible with the presence of physiological modules at the interface of the associative—cognitive—cortex with the environment. In fact, the present model assumes sensory and motor modules at the foundation of memory networks. However, with those modules at the base, the architecture of the present network model takes the form of a massive scaffolding of hierarchically organized memory networks in a continuum of increasing network size, from the primary cortex to the highest levels of association cortex.
That the cortex in its totality is a network is a truism. What is far from a truism is the parceling of that gigantic network into the multiplicity of overlapping, interactive, and specialized memory networks that emerges from the recent studies compiled in this review. The emerging model postulates that the neocortex harbors an immense array of distinct neuronal networks dedicated to the representation and retrieval of individual memory and knowledge. Still largely unknown, however, are the structure and the dynamic properties of those networks, including their mechanisms, their resilience in the face of neural damage, their deterioration in disease and old age, and their potential for rehabilitation.
As noted, the present network model is built upon a modular model. Sensory modules, conceivably with simple netlike structure, represent simple sensory stimuli associated in evolution with others with similar features (see later, phyletic memory). Complex sensory stimuli of the same or different modality associate those simple sensory networks into larger networks of association cortex. Those, in turn, form even larger networks to represent yet more complex perceptual information. Thus, memory networks of increasing amplitude come to represent progressively more complex perceptual memories in progressively higher levels of posterior cortex. In sum, a hierarchy of increasingly wider networks develops there to represent a hierarchy of progressively higher and more complex memories and knowledge, from sensory qualia at the bottom to semantic and conceptual memories at the top. Arguably, a comparable hierarchy develops in frontal cortex to represent motor, “procedural,” and executive memories and knowledge. However, at some stage in the hierarchies from sensory and motor cortices into association cortex, the present network model departs radically from other modular or network models of memory in four fundamental ways:
In the present model, a memory or an item of knowledge consists of a widespread cortical network of connections, formed by experience, that joins dispersed cell populations. These cell populations represent the associated percepts and actions that, together, constitute that memory or cognitive item. Thus, the memory code is fundamentally a relational code, sparse and distributed, etched in cortical space by connections between distributed neurons—unlike the information in a theoretical “module of memory.”
A complex memory network, such as an autobiographical memory, is largely interregional, linking neuron assemblies and smaller networks in separate and noncontiguous areas of the cortex; in turn, those assemblies or networks represent other, more concrete aspects of memory or knowledge.
As a result of the practically infinite combinational power of billions of cortical neurons, memory networks differ widely in content, complexity, source, temporal origin, and level of abstraction—from concrete sensation or action to semantic or conceptual knowledge and plans of action. Accordingly, the individuality of memory derives from that combinational power.
Memory networks overlap and interlink profusely with one another by common nodes (i.e., smaller networks), whereby a cortical neuron or neuronal assembly, practically anywhere in the cortex, can be part of multiple networks, thus of multiple memories or items of knowledge.
These architectural features distinguish this network model from the more conventional modular and network models of cortical memory, making the transition from those models to the present one a shift of scientific paradigm à la Kuhn (1996). The principal purpose of this review is to critically examine and substantiate those four tenets.
Lashley (1950), after unsuccessfully attempting to induce memory deficits by discrete cortical lesions, inferred widely distributed memory, almost by default. At about the same time, others (Hayek, 1952; Hebb, 1949) began to postulate cortical network models of perception, learning, and memory. Several neuroscientists subsequently incorporated variants of the network idea in their theoretical constructs of cortical cognition (McIntosh, 2000; Mesulam, 1998; Bressler, 1995; Goldman-Rakic, 1988; Edelman & Mountcastle, 1978; Luria, 1966). Further theoretical support for that idea came from the fields of neurocomputation and artificial intelligence (McClelland & Rumelhart, 1986; Hinton & Anderson, 1981), especially connectionism (Marcus, 1998; Fodor & Pylyshyn, 1988; Myers, 1967).
Not until recently, however, has a flood of empirical evidence given to the network memory paradigm here presented its innovative and distinctive character. The evidence comes from three confluent methodologies: microelectrode recording in the behaving primate, computational analysis of electrocortical potentials, and functional imaging in the human. The three methodologies provide insight into the structure and dynamics of memory networks. Elsewhere (e.g., Fuster, 2003), I have used the term cognit to characterize a memory network because such a network can represent semantic knowledge as well as autobiographical memory, with comparable network structure and the same essential features noted above. In this review, the two terms, cognit and memory network, are used interchangeably.
STRUCTURE OF A MEMORY NETWORK (COGNIT)
Any reasonable model of cortical memory must accommodate two interrelated phenomenological facts: the heterogeneity and the integrative character of memory. Theoretically, any memory network is heterogeneous because it includes or can include semantic facts as well as events, categories as well as sensory qualia, percepts as well as actions, and biological incentives as well as value principles. Thus, taxonomies of memory by content are not very helpful to the cognitive neuroscience of cortical memory. More helpful is the evidence that the categories of perceptual memory predominate in posterior cortices, whereas those of executive memory predominate in frontal cortices. This review, however, points to the distributed and intermixed character of most all cortical memory. In the light of that evidence, the specific localization of complex knowledge, such as autobiographical memory, to single neurons or neuron assemblies is theoretically and empirically implausible. That does not deny the presence of certain cortical foci of heavy association (network nodes) that by lesion or stimulation are implicated more in one type of memory than in others.
The other obligatory attribute of memory, which the new paradigm accommodates, is its integrative character. In static as well as dynamic terms, all memory is essentially associative, in its formation, in its structure, and in its activation. Associative integration is essential to acquisition, storage, and recall, especially if the memory or item of knowledge is complex. Take away integration, and the memory degrades or literally disintegrates. The same is true for knowledge and semantic memory; in their case, however, the disintegration is less likely because they are anchored in multiple and to some degree redundant cortical associations (i.e., connections).
Integration is missing in most modular models of memory or perception, even in those that are based on a hierarchical architecture similar to that of the network model discussed here. Two constraints make their architecture orthogonal to that of this model: absence of associative connections at low levels (e.g., cross-modal, sensorimotor) and absence of heterarchy (see below) in the associated networks and assemblies. Further, many modular models are based on the assumed hierarchical organization of the visual system, which they expand to visual perception. That assumption almost inevitably leads to the theoretical absurdity of the “grandmother cell.” The network paradigm proposes almost the opposite. Although allowing for some convergence of connections into certain relatively “specialized” regions (e.g., for faces, words), this paradigm emphasizes the divergence of connection toward the top, toward ever wider networks that represent ever more categorical and abstract information; thus, integration takes place everywhere, but especially at the top, among widely dispersed elements.
In the present network paradigm, a cognit is defined by associations and connectivity and is thus essentially an integrative entity. Two currents of cognitive science are closely related to it. One is connectionism, the other Gestalt psychology. Although the first was originally a doctrine that based all behavior on the bonds between stimuli and responses, it later developed into a cognitive theory of the relationships between inputs and outputs in the processing of neurocognition (Marcus, 1998; Fodor & Pylyshyn, 1988; Myers, 1967). It has been applied to modular formulations of language and other complex cognition.
Our network paradigm is even more germane to Gestalt psychology. The latter began as a theory of visual perception (Koffka, 1935), according to which a form or gestalt is defined by the mutual relationships between its parts and irreducible to them. That theory postulates a set of rules about the nature of those relationships (cohesion, continuity, similarity, etc.). Eventually, Gestalt psychology transcended visual perception and the trite dictum that the whole is more than the sum of the parts. Nonetheless, it has practically disappeared from current discourse in cognitive science. One significant legacy of Gestalt, however, is the tenet that perception is based on a relational code. Perception, Hayek (1952) adduced, is the classification of the world by an isomorphic set of connective relationships previously established in the cortex by temporal coincidence of external stimuli, hence the inextricable relationship between perception and memory (as first stated by Helmholtz, 1925), which led Hayek to postulate overlapping cortical “maps” of perceptual experience, much like the memory networks or cognits postulated here.
Figure 1 illustrates in highly schematic and three-dimensional manner the chief principles of network architecture in one cortical hemisphere. Cognitive networks or cognits are represented in the figure by circles of different size depending on network size and hierarchical level. At lower levels (sensory and motor cortices, “deep” in the figure), cognits are small, their networks within the limits of traditional sensory and motor “modules.” At higher levels, in cortex of association, cognitive networks are larger and represent broader categories of memory (polymodal, semantic, episodic, etc., perceptual or executive), which are distributed in the cortex within broad areas or across regions. Cognits of almost any size and level can constitute the nested nodes of more widely distributed networks. Some such networks are large cognits that represent complex memories or items of knowledge of different hierarchical level; in other words, they are heterarchical cognits. Two such networks are shown activated in Figure 1B and C.
The present network model differs distinctly from other models of associative nets (e.g., Hinton & Anderson, 1981). Its two principal distinguishing characteristics are as follows:
Hierarchy of memory networks and contents. Although the present module is layered like most other network models, it accommodates a hierarchy of networks and contents whereas other models do not.
Nets with common nodes. The present model postulates that memory networks share common associative nodes—smaller cognits, with network structure themselves—that represent contents common to several nets.
Some of the most valid inferences about the structure of a cognit can be drawn from its dynamics in behavior. It is by studying a memory network in the active state that we can glean its structure. Memory networks are activated in a variety of conditions. First and foremost, some are activated, consciously or unconsciously, voluntarily or involuntarily, in every act of perception. They are also activated, more or less voluntarily, in free recall, recognition, new memory acquisition, rehearsal, and working memory. The last, I argue, is the most suitable condition for the study of active memory networks.
Working memory is the ability to temporarily retain information for a prospective action. The reasons for its suitability to the study of memory structure and dynamics are because (1) it is relatively easy to instruct a human or an animal to retain, for a defined period, a specific item of information that calls for a specific prospective action; (2) it is reasonable to assume that, during that time, the neural substrate for mnemonic retention is in a different state than at rest; (3) the physiological measures of differing state probably reflect the physical nature of the information in temporary storage, in such a manner that changes in the parameters of that information will be reflected by commeasurable changes in network dynamics; and (4) the extent of those changes in the cortex and their correlations with memory performance will help us determine the boundaries and the dynamics of a cognit.
For nearly 40 years, it has been known that in the monkey performing working-memory tasks, such as delayed response and delayed matching, neurons in certain cortical regions undergo persistent elevations of firing frequency during working memory. Because they were assumed to intervene in a memory process, such neurons were named “memory cells.” Ordinarily, these cells show a transient frequency change in response to the sensory cue that the animal must retain in memory. After the cue has disappeared, that change is followed by above-baseline discharge for much or all of the ensuing delay or memory period—seconds or minutes—before the motor response or choice.
Memory cells were first encountered in the pFC of monkeys performing delayed response (Fuster & Alexander, 1971). Subsequent investigations of that cortex revealed memory cells whose level of memory activity was dependent on the physical characteristics of the stimulus cue to be remembered (memorandum). Some prefrontal cells respond preferentially to spatial memoranda (Genovesio, Brasted, & Wise, 2006; Constantinidis, Franowicz, & Goldman-Rakic, 2001; Funahashi, Bruce, & Goldman-Rakic, 1989; Niki, 1974), others to visual memoranda (Wilson, Scalaidhe, & Goldman-Rakic, 1993; Fuster, Bauer, & Jervey, 1982), including faces (Scalaidhe, Wilson, & Goldman-Rakic, 1999), and still others to auditory (Romanski, 2007; Fuster, Bodner, & Kroger, 2000) or tactile (Romo, Brody, Hernández, & Lemus, 1999) memoranda. Furthermore, in some studies, the relation between prefrontal memory firing and physical stimulus dimension has been found parametric for such properties as the location of gaze (Funahashi et al., 1989) or the frequency of mechanical vibration sensed by touch (Romo et al., 1999). In any given prefrontal area, however, the memory cells that show sharp tuning for any given sensory memorandum constitute a minority.
Cells preferring memoranda of one sensory modality or another tend to concentrate in certain domains of pFC, but these domains are poorly demarcated; cells with any given preference can be found practically anywhere in pFC. For example, cells preferring spatial memoranda predominate in dorsolateral areas but are also present in ventrolateral areas, although in lower numbers, and vice versa for nonspatial memoranda. Likewise, cells preferring and anticipating a reward concentrate in an orbital domain but are also present in lateral cortex. Some cells are attuned to stimuli of more than one sensory modality—for example, auditory and visual—that have become associated with one another by the learning of a working-memory task (Artchakov et al., 2007; Romanski, 2007; Fuster et al., 2000; Watanabe, 1992; Vaadia, Benson, Hienz, & Goldstein, 1986). Others are attuned to both the spatial and the nonspatial attributes of the memorandum (Fukushima, Hasegawa, & Miyashita, 2004; Rao, Rainer, & Miller, 1997; Fuster et al., 1982).
Furthermore, in any prefrontal region, some cells are also attuned to the motor requirements of the working-memory task (Isomura, Ito, Akazawa, Nambu, & Takada, 2003; Akkal, Bioulac, Audin, & Burbaud, 2002; Procyk & Joseph, 2001; Quintana & Fuster, 1999; Carlson, Rämä, Tanila, Linnankoski, & Mansikka, 1997; Fuster et al., 1982) and/or the reward expected or resulting from motor action (Ichihara-Takeda & Funahashi, 2008; Hikosaka & Watanabe, 2000; Schultz, Tremblay, & Hollerman, 2000; Watanabe, 1996; Rosenkilde, Bauer, & Fuster, 1981). The prefrontal domains with relatively high concentration of cells attuned to different modalities, or task attributes are anatomically connected with specialized posterior cortical areas or subcortical structures (review in Fuster, 2008). Visual and auditory domains of ventral pFC are connected with corresponding visual and auditory areas of temporal cortex and reward domains of orbital cortex with tegmental and limbic formations.
The memory cells in posterior association cortex generally show more sensory specificity than those in pFC and tend to cluster in areas of association for the sensory modality of the memorandum. Thus, cells in inferotemporal cortex (Figure 2) are tuned to visual memoranda (Miller, Li, & Desimone, 1991; Miyashita & Chang, 1988; Fuster & Jervey, 1982), whereas cells in parietal cortex are tuned to spatial (Andersen, Bracewell, Barash, Gnadt, & Fogassi, 1990) or tactile (Zhou, Ardestani, & Fuster, 2007; Burton & Sinclair, 2000; Zhou & Fuster, 1996) memoranda. As in the pFC, although to a lesser degree (in terms of cell numbers and response magnitude), cells can be found in posterior areas that respond to the motor and/or reward attributes of memory tasks.
The thus far summarized microelectrode evidence points to the widely distributed character of cortical memory networks, with components in posterior (sensory or perceptual) cortex and in frontal (executive) cortex (Constantinidis & Procyk, 2004; Fuster, 1995). It further points to the presence of cells and cell assemblies that respond to more than one memorandum or different characteristics of the same memorandum. This property of “multiple tuning” argues for the belonging of cells to multiple networks and for their flexible functional allegiance to those networks. Next, I discuss how single-unit evidence substantiates the associative character of working-memory networks and their structural identity with long-term memory networks.
WORKING MEMORY FROM LONG-TERM MEMORY
The neuropsychological literature provides indirect evidence that the cortical substrate for working memory coincides with the substrate for long-term memory. That evidence derives from a large number of animal and human lesion studies (reviewed in Fuster, 1995), indicating that several cortical areas are implicated in the working memory of certain classes of stimuli as well as in their retention in long-term memory. The lesion of a given associative area in the monkey induces deficits in delayed-response tasks with stimuli of a given modality (visual, spatial, tactile, etc.), as it does also in the recall or recognition of stimuli of the same modality. In the human, amnesias or agnosias for that modality result from lesions of areas roughly homologous to those of the monkey that are related to the same modality.
By definition, moreover, working memory determines a prospective action, commonly a decision between alternatives (e.g., a selective instrumental response, a verbal response, a logical inference, etc.) that is based on long-term memory. Its content and context are inextricably linked to that memory, whether they include a simple learned conditional response to a simple sensory stimulus, an element of lexicon, or the conclusion of a syllogism. Thus, whereas most long-term memory never enters working memory, all working memory is not only anchored in long-term memory but also part of it. That part of long-term memory varies with the circumstances and is evoked by them; that is, by the associations of the material to be retained and the prospective consequences of that act of retention, all of them previously stored in permanent memory.
More direct evidence of the associative nature of working-memory networks, and thus their identity to cognits of long-term memory, comes from electrophysiology. Especially demonstrative are memory cells that show correlated responses to stimuli of the same or different modality associated with one another by learning. Such cells have been found in both frontal (Fuster et al., 2000) and posterior (Schlack & Albright, 2007; Zhou et al., 2007; Messinger, Squire, Zola, & Albright, 2001; Gibson & Maunsell, 1997; Miyashita, 1988) association cortices (Figure 3); tactile–visual cell response correlations can be found even in primary somatosensory cortex (Zhou & Fuster, 1995). The evidence becomes compelling as the correlations develop in the process of acquiring the associations in long-term memory. Further evidence of the identity of working-memory and long-term memory networks is the observation that working-memory cells do not respond only to the memoranda but also to assorted sensory and motor inputs that are associated with the performance of the task at hand (Zhou et al., 2007). In visual processing, the associative inputs from other parts of the sensorium would contribute to a stimulus of what has been considered context (Bressler & McIntosh, 2007; Albright & Stoner, 2002; Fuster, 1990). Context includes the history and the meaning of the stimulus in long-term memory.
Whereas sensory–sensory associations in working memory have been demonstrated in both posterior and frontal cortex, sensory–motor and motor–motor associations appear mainly, if not exclusively, in pFC. There, cells seem to belong to assemblies that encode behavioral sequences or “temporal motor gestalts” of relatively abstract nature (Averbeck & Lee, 2007; Shima, Isoda, Mushiake, & Tanji, 2007), regardless of their individual motor components (Figure 4). Furthermore, some frontal memory cells are attuned to conditional rules, another indication of their involvement in associative long-term memory (Mansouri, Matsumoto, & Tanaka, 2006; Wallis, Anderson, & Miller, 2001; Asaad, Rainer, & Miller, 2000; White & Wise, 1999).
The associative character of working-memory cells substantiates the assumptions (a) that memory cells belong to networks of long-term memory that associate the various sensory and motor features of working-memory demands, including but not limited to the memoranda used by the animal in working-memory tasks, (b) that those networks can incorporate widely dispersed neurons in noncontiguous cortical areas, and (c) that the networks and their associations are formed by experience.
Within a given cognitive network, however, the density of associations appears far from homogeneous. To judge from the groupings of cells attuned to the same or similar stimuli, a working-memory network seems to contain nodes of heavy association in areas of relative functional specialization. Such nodes may be characterized as special “modules” or “mini-networks” for spatial, visual, auditory, tactile, or other working memory. However, their function in the retrieval of long-term memory, as in working memory, is strictly dependent on their previously established associations with other modules or cells scattered over the width and the depth of the cortex of association, both frontal and posterior.
To sum up, studies of cortical memory cells lead to the tentative conclusion that working memory consists of temporary activation of a preexistent network of long-term memory. That network connects with one another all the smaller networks and neuronal assemblies that, in the aggregate, represent the associated features, perceptual as well as executive, of the behavioral interaction of the organism with its environment. At the onset of a trial in a working-memory task, the activated network is updated by the presence of the memorandum. Thus, the updated network of long-term memory becomes operational, and the process of temporary retention of that memorandum begins within the context of the task. Neither the content nor the dynamics of working memory can be separated from the substrate of a sector of long-term memory and its temporary activation to bridge the recent past with the proximate future.
MECHANISMS OF WORKING MEMORY
The mechanisms of persistent neuronal activation in working memory are not well understood. One key mechanism appears to be the reverberation of cortical circuits by reentry of activity from one neuron or neuronal assembly to another by mutual excitation. That mechanism was first proposed by Hebb (1949) as the basis of short-term memory. It has never been conclusively proven, although now it is gathering evidence as the basis of working memory.
Cortical reverberation by reentry has a well-recognized anatomical base. The existence of profuse recurrent axons in the cortex has been known since Lorente de Nó (1949) described them; the axons make reciprocal excitatory synapses on neighboring or distant neurons. Most all pathways between frontal and posterior association cortices are bidirectional, and so are the connections between those cortices and thalamic nuclei (Petrides & Pandya, 2002; review in Fuster, 2008). Therefore, there is ample structural potential in cortical circuitry for sustained reverberation in working-memory networks, however separate their neurons may be. Accordingly, reentry is an integral part of the most plausible computational models of working memory (Warden & Miller, 2007; Wang, 2006; Brunel & Wang, 2001; Zipser, Kehoe, Littlewort, & Fuster, 1993). Some of these models assume that reentry connections are mediated through dopamine, glutamate, γ-aminobutyric acid, or N-methyl d-aspartate synaptic transactions.
Most computational models of working memory, however, assume the bistability of their functional architecture: discharge at a certain fixed elevated frequency when in memory and reversion to baseline frequency when not. This assumption does not agree with data from the real brain. Already from early observations (e.g., Fuster, 1973), it has been known that, if the delay (memory period) is long enough (>10 sec), the time course of memory activity can adopt many forms. The most obvious are the descending ramps of sensory-coupled cells at the start of the delay or the accelerating ramps of motor-coupled cells during it (Figure 5), but in any case with considerable variability from cell to cell and from trial to trial (Shafi et al., 2007). Many memory cells show mixed temporal patterns. The analysis of interspike intervals (ISIs) reveals, in both the baseline (intertrial) and the delay (memory) periods, a multiplicity of ISI patterns independent of frequency (Bodner, Shafi, Zhou, & Fuster, 2005). In working memory, the number of observable patterns generally increases.
The multiplicity of firing patterns in working memory is fully consistent with the notion that memory cells belong to multiple networks (Figure 1B and C). One possibility is that, when the cortex enters working memory, its memory cells fire at a variety of frequencies, each frequency the expression of activity in a given reentry loop transiting through those cells. The cells seem recruited into a variety of reentry loops, each loop with its own reverberating frequency. It is reasonable to infer that the cells belong to the multiple associative networks that define the various attributes of the memorandum—including its prospective attributes related to action and reward. Those networks can be local or far-flung. Depending on the attribute in the focus of working memory at any given time, a cell would change its allegiance to one network or another and thus become attuned to one reverberating frequency or another.
Whether the reentry is local or far-flung, persistence of working-memory discharge may also be caused at least in part by long-lasting ionic changes at synaptic level. A recent model (Mongillo, Barak, & Tsodyks, 2008) postulates that working memory is sustained in recurrent neuronal connections by residual calcium released by incoming spiking, such as that elicited by the memorandum. That residual calcium would increase presynaptically, thereby facilitating synaptic transmission and responses to subsequent inputs, including further recurrent inputs or replicates of the memorandum (Figure 6). To maintain active memory, the model would economize on spikes. Also, the following two characteristics make the model fully compatible with our network assumptions: (1) reentry within recurrent cortical networks and (2) preexistent memory code presumably established by Hebbian associative learning and activated by the memorandum. Because of their multiple interactions, several networks (long-term associative memories) can be activated at the same time (Figure 6B).
Corticocortical circuitry can maintain memory active not only by reverberation but also by tonic influences of one area upon another or from one part of a cognit upon another. That circuitry would involve the well-known anatomical connections between prefrontal and posterior cortices (Barbas, Ghashghaei, Rempel-Clower, & Xiao, 2002; Petrides & Pandya, 2002; Jones & Powell, 1970). The importance of mutual influences between those cortices for memory maintenance is revealed by the results of the functional inactivation—by cooling—of pFC on the activity of memory cells in posterior cortex or vice versa (Chafee & Goldman-Rakic, 2000; Quintana, Fuster, & Yajeya, 1989; Fuster, Bauer, & Jervey, 1985). For example (Fuster et al., 1985), cooling the lateral pFC to 20°C during a visual delayed match-to-sample task diminishes the ability of some inferotemporal cells to discriminate the color of the memorandum during the memory period (delay); presumably as a consequence, the animal commits more errors of color working memory than at normal cortical temperature. Thus, prefrontal cooling diminishes “cognitive control” (Miller & Cohen, 2001) over the inferotemporal cortex in working memory. Alternatively (or concomitantly), the procedure interrupts the reverberating loop between the two cortices that presumably supports active memory maintenance.
In sum, the mechanisms of working memory are not yet established, but there is increasing evidence that they include the reverberation in circuits within the neural network, part of long-term memory, that has been activated by the memorandum. Additional evidence is to be found in the study of patterns and periodicities of memory-cell firing.
RHYTHM AND SYNCHRONY
The presence of electrical oscillations in the cortex has been well known since Hans Berger (1929) described the EEG on the head's surface. Oscillations are encountered not only in the extracranial EEG but also in the local field potentials (LFPs), which are intracortical signals reflecting summed dendritic activity in local neuronal assemblies; essentially, LFPs have the same electrogenesis as the EEG but in more discrete cortical domains than those that generate the EEG signal. Many rhythms have been related to a wide variety of brain states and psychological conditions. Although their observable range is almost continuous, those rhythms can be categorized by frequency into a finite number of discrete classes (Figure 7). The physiological exploration of electrocortical rhythms is especially productive with regard to two highly interrelated cognitive functions: perception and attention. Both these functions are, in turn, highly related to memory. Perceiving is remembering as much as sensing, and working memory is attention focused on an internal representation.
In the late 1980s, it was discovered that separate groups of neurons in the visual cortex fired in synchrony (gamma range) in response to visual stimuli (Jagadeesh, Gray, & Ferster, 1992; Engel, König, Kreiter, & Singer, 1991; Gray & Singer, 1989; Eckhorn et al., 1988). One interpretation of this phenomenon was that separate neurons firing in synchrony engaged in what was called “perceptual binding,” that is, the binding of the diverse visual features of the stimulus into a perceptual whole (Engel, Fries, & Singer, 2001; Gray, Konig, Engel, & Singer, 1989). Neural binding, thus, became a model of perceptual constancy toward solving one of the central problems of physiological psychology (Klüver, 1933). Perceptual constancy would emerge from the binding of sensory features, namely, from the mutual functional relationships between cell groups despite physical variations in each of those features individually. A relational code would thus emerge in similar manner as a gestalt. Could oscillations play the same role at higher cognitive levels, especially in working memory?
The causal relationships between electrocortical oscillation and cell spiking are obscure, although there is increasing evidence of the coupling of cell spikes and LFPs (Lee, Simpson, Logothetis, & Rainer, 2005; Pesaran, Pezaris, Sahani, Mitra, & Andersen, 2002; Fries, Reynolds, Rorie, & Desimone, 2001). On the one hand, periodic spike trains can generate periodic dendritic potentials (Reyes, 2003). On the other, dendritic oscillations can change the excitability of a cell's membrane, thereby biasing its production of spikes, which may occur in phase at various times with respect to the oscillatory cycle (Tsodyks, Skaggs, Sejnowski, & McNaughton, 1996). Both mechanisms may operate in the cortex to some degree, in any case leading to temporal correlations between periodic spikes and oscillations.
Synchronous cortical oscillations appear to result from the interaction of both, local factors at the membrane of the cell and circuit factors at the network in which the cell is embedded. Probably among the latter are the periodic inputs from the thalamus (Steriade, 2001; Llinás, 1988), the so-called “inhibitory clocking networks” of interneurons (Buzsáki, Geisler, Henze, & Wang, 2004), and the loops of corticocortical connections. In ensuing discussion, I emphasize the last of these factors for its relevance to reentry in working memory, without excluding a coadjutant or even primary role for the others.
If we assume that reentry is an important component of network architecture, length of circuitry should be a determinant of oscillatory frequency. However, length of circuitry does not necessarily mean length of fibers. Braitenberg and Schüz (1998) argued that some long fibers (e.g., corticocortical) may actually shorten the effective connectivity within networks. It follows that there could be an inverse relationship between network size, in terms of effective circuitry, and frequency of oscillation (Buzsáki & Draguhn, 2004; Freeman, Rogers, Holmes, & Silbergeld, 2000). This reasoning accommodates a good amount of electrophysiological data (Buzsáki et al., 2004; Csicsvari, Jamieson, Wise, & Buzsaki, 2003; Steriade, 2001).
Complex cortical processing during behavior involves many networks, large and small, some nested within others. This, in conditions of heightened attention and working memory, will entail a proliferation of activated networks oscillating at multiple frequencies. More generally, the proliferation and the fragmentation of frequencies are most likely to be at the root of the “desynchronization” of the EEG as the subject awakes from sleep or responds to sensory stimuli (Basar & Bullock, 2000; Pfurtscheller & Lopes da Silva, 1999). By contrast, at rest or in sleep, when simple large networks prevail, low frequencies (theta or lower) will predominate. This does not preclude that, by virtue of their stereotypical functional architecture, certain parts of “ancient cortex” involved in memory, such as the hippocampus, exhibit low oscillatory frequencies even, or especially, when its networks are highly active (Huxter, Burgess, & O'Keefe, 2003; Penttonen & Buzsáki, 2003; Huerta & Lisman, 1995).
In the sensory association cortex engaged in sharply focused (“top–down”) attention to the location or characteristics of sensory stimuli, high-frequency oscillations (beta and gamma) have been observed in LFPs as well as in unit discharge (Lakatos, Karmos, Mehta, Ulbert, & Schroeder, 2008; Buschman & Miller, 2007; Saalmann, Pigarev, & Vidyasagar, 2007; Womelsdorf, Fries, Mitra, & Desimone, 2006; Brovelli, Lachaux, Kahane, & Boussaoud, 2005; Fries et al., 2001). Presumably, those oscillations reflect reentrant activity in the small, high-frequency oscillating networks that process the item of information that the subject attends to. That phenomenon is more apparent when attention is extended in time. This is the situation in working memory, which is attention focused on the internal representation of a recent stimulus for prospective action (Fuster, 2003; Baddeley, 1993). During working memory, in sensory or association cortex, oscillatory synchrony (Figure 8) commonly appears (Lee et al., 2005; Rizzuto et al., 2003; Pesaran et al., 2002; Tallon-Baudry, Bertrand, & Fischer, 2001). In the human, synchrony predominates in the beta and theta ranges. Further, the desynchronizing transition from alpha to beta or higher has been noted to be selective in different areas depending on the modality or memory load of the memorandum (Stipacek, Grabner, Neuper, Fink, & Neubauer, 2003; Klimesch et al., 1996).
In higher association cortex, cellular activity during memory activation is multistable and multivariate, as networks there profusely intersect. They represent in long-term memory the multiple associated aspects of the memorandum. Accordingly, and supporting our reasoning for the commonality of anatomical substrates for working and long-term memory, we have observed cell-firing frequencies attuned to several associated aspects of a working-memory task (Zhou et al., 2007; Fuster et al., 2000). During the memory period, the analysis of ISIs shows extensive variability (Shafi et al., 2007) and a proliferation of patterns (Bodner et al., 2005), both supposedly reflecting the affiliation of cells to multiple memory networks. Thus, the electrocortical records from human subjects in situations of high cognitive demand exhibit a multiplicity of rhythms in areas of association. Especially prevalent are oscillations in the theta, alpha, beta, and gamma frequencies (Sehatpour et al., 2008; Gevins & Smith, 2000), in some instances modulating one another (Lakatos et al., 2008; Canolty et al., 2006).
The most direct electrocortical evidence of the activation of interregional networks in working memory is the synchrony of high-frequency (beta and gamma) oscillations in frontal and posterior regions of association cortex during high attention and working-memory performance (Axmacher, Schmitz, Wagner, Elger, & Fell, 2008; Sehatpour et al., 2008; Buschman & Miller, 2007; Saalmann et al., 2007; Brovelli et al., 2004; Gross et al., 2004; Brovelli, Battaglini, Naranjo, & Budai, 2002; Tallon-Baudry et al., 2001; Stein, Rappelsberger, Sarnthein, & Petsche, 1999; Bressler, Coppola, & Nakamura, 1993). In the next section, we see in neuroimages the reflection of that synchrony in posterior and frontal regions during working memory. In any case, taken as a whole, the electrocortical evidence strongly supports the broader principle of widely distributed and overlapping memory networks.
NEUROIMAGING OF MEMORY-NETWORK ACTIVATION
The judicious use of functional imaging methods in the human has contributed mightily to support the network paradigm of memory. It has also contributed to the understanding of the dynamics of memory networks. However, the imaging methodology has considerable limitations. It is essential to be aware of them before any review of imaging data on memory.
The following are the most relevant limitations of PET and fMRI: (1) The neurovascular coupling function is still poorly understood. (2) Temporal resolution is inadequate to measure rapid changes in memory acquisition and recall. (3) Large individual variability limits conclusions on memory distribution or mechanisms. (4) Linear models of blood-flow change may not be fully compatible with memory functions inherently nonlinear. (5) To the extent that memory uses the same cortical networks as other related cognitive functions (e.g., attention, perception, language), it is difficult to disambiguate memory activation from that of those other functions. And (6) the neural inhibition of memory cannot be easily differentiated, by imaging, from its activation. Because of these limitations, the merits of any imaging study of memory depend on the investigator's ability to make only indispensable assumptions and use appropriate controls and analytical methods. The evidence summarized in this section comes from studies that meet those criteria.
Neuroimaging does not allow the precise tracing of the boundaries of active memory networks. The conventional assessment of cortical activation, in both intensity and extensity, is essentially analogous to that of separating signal from noise. The investigator sets a threshold—calculated from normalized baseline values—and determines the significant deviations that exceed it. Here the problem is to distinguish the variance of active memory from that of background noise. The presence of “default networks,” active in the resting state (Fox & Raichle, 2007), as well as the increased variance and the diminished activation at the edges of an active memory network makes the threshold setting critical yet somewhat arbitrary. If that threshold is too low, the networks appear larger than they are; if it is too high, the networks appear smaller.
Like microelectrode research, the functional neuroimaging of working memory focused at first on the pFC, that is, on the executive sector of memory networks. PET studies showed activation of lateral pFC, especially on the right, in spatial working memory (Jonides et al., 1993; Petrides, Alivisatos, Evans, & Meyer, 1993). That was subsequently corroborated by fMRI studies (Ricciardi et al., 2006; Ranganath, Cohen, Dam, & D'Esposito, 2004). Both PET (Swartz et al., 1995) and fMRI (Cohen et al., 1994) showed also lateral prefrontal activation in visual nonspatial working memory. A special case of the latter is the memory for faces, which activates not only the lateral prefrontal areas but also the cortex of the fusiform gyrus, an area involved in face recognition (Rama & Courtney, 2005; Gazzaley, Rissman, & D'Esposito, 2004; Ranganath et al., 2004; Mecklinger, Bosch, Gruenewald, Bentin, & Von Cramon, 2000). The pFC, especially on the left, is activated in verbal working memory (Buchsbaum, Olsen, Koch, & Berman, 2005; Goldstein et al., 2005; Narayanan et al., 2005; Crottaz-Herbette, Anagnoson, & Menon, 2004; Paulesu, Frith, & Frackowiak, 1993; Petrides, Alivisatos, Meyer, & Evans, 1993) and also in working memory for mental arithmetic (De Pisapia, Slomski, & Braver, 2007; Kondo et al., 2004).
Whatever the content in working memory, the amount of prefrontal activation is directly proportional to the memory load (Narayanan et al., 2005; Cairo, Liddle, Woodward, & Ngan, 2004; Leung, Seelig, & Gore, 2004; Jaeggi et al., 2003; Linden et al., 2003; Postle, Berger, Goldstein, Curtis, & D'Esposito, 2001). In other words, prefrontal activation increases as a function of the number and complexity of items in memory. Practice, however, decreases load-related activation. Is this a sign of memory consolidation, which entails economy of synaptic resources? Or, is it of the migration of executive memory to subcortical structures (e.g., basal ganglia)?
Practically all the relevant studies show that the pFC is not the only region activated in working memory. Almost invariably, one or more posterior cortical areas are concomitantly activated. Which posterior area or areas are activated depends on the modality of the memorandum: inferior temporal areas if it is visual (additionally fusiform cortex, if it is a face), posterior parietal if it is spatial, superior temporal if it is auditory or verbal, and anterior parietal if it is tactile. Homologous areas of cortex appear activated in imaging records of the human as in microelectrode records of the monkey. Prefrontal areas are activated inasmuch as executive memory is involved and posterior areas inasmuch as perceptual memory is involved.
Consistent with the microelectrode evidence of interactions between prefrontal and posterior association areas, several imaging studies indicate that those interactions underlie the role of pFC in so-called “executive cognitive control” or “top–down attention” (Roth, Serences, & Courtney, 2006; Yoon, Curtis, & D'Esposito, 2006; Buchsbaum et al., 2005; Curtis, Sun, Miller, & D'Esposito, 2005; Postle, 2005; Kondo et al., 2004; Li et al., 2004; Sakai & Passingham, 2004). The medial pFC—especially anterior cingulate—seems part of a so-called “anterior attention system,” dedicated to spatial attention (Lenartowicz & McIntosh, 2005; Kondo et al., 2004; Petit, Courtney, Ungerleider, & Haxby, 1998; Posner & Petersen, 1990). However, the attribution of “control” to the pFC, in attention or any other cognitive function, implies for that cortex a role of “central executive,” which makes little sense in biological terms and leads to an infinite regress (Fuster, 2008; McIntosh, 2000).
In the light of imaging data, the “central executive” role of the pFC is in principle reducible to its role of integrating for prospective action a continuous flow of inputs from the internal and the external environments. The memory networks of posterior cortex are part of the internal environment, which in turn can be activated by external stimuli and the effects of action, all within the perception–action cycle (below). In that same framework, it is possible to understand the role that imaging studies attribute to the pFC in the retrieval and encoding of memory (Mitchell, Johnson, Raye, & Greene, 2004; Ranganath et al., 2004; Rypma & D'Esposito, 2003; Lee, Robbins, & Owen, 2000; Buckner et al., 1995; Kapur et al., 1994; Tulving, Kapur, Craik, Moscovitch, & Houle, 1994; Tulving, Kapur, Markowitsch, et al., 1994); both retrieval (except in involuntary or free recall) and encoding are executive acts prompted by external stimuli.
Figures 9,1011–12 illustrate schematically the trends of cortical activation on the left hemisphere during performance of three working-memory tasks: visual, spatial, and verbal. The activation images consist of stills extracted from motion pictures constructed by graphic synthesis of data in the following publications (those preceded by an asterisk are based on quantitative meta-analysis of multiple studies): Buchsbaum et al., 2005; Goldstein et al., 2005; *Rajah & D'Esposito, 2005; Crottaz-Herbette et al., 2004; *Wager & Smith, 2003; *Cabeza & Nyberg, 2000; D'Esposito, Postle, & Rypma, 2000; *Duncan & Owen, 2000; Mecklinger et al., 2000; Pollmann & Von Cramon, 2000; *Casey et al., 1998; Petit et al., 1998; and Courtney, Ungerleider, Keil, & Haxby, 1997. No attempt was made to normalize quantitative differences. The time course of activation, which is unavailable in the majority of publications, was grossly estimated based on unit data from the primate in working-memory tasks.
A reasonable explanation of the joint prefrontal–posterior activation and functional interdependence in working memory is that in the course of behavior—as in reasoning and language—a prefrontal network of executive memory interacts with a posterior network of perceptual memory. Both complement each other and cooperate in short-term, long-term, and working memory. Both control each other reciprocally at the top of the perception–action cycle. In serial behavior, the control shifts successively in circular fashion between the two. In working memory, that reciprocal interaction adopts the form of neural reverberation between—and within—the two to retain the memorandum and its associations, including the expected response and reward.
In summary, imaging shows that working memory activates simultaneously a region of pFC and at least one other region of posterior cortex. As indicated by the other reviewed methodologies, reverberating reentry between the two, at the top of the perception–action cycle, is probably the key mechanism of working-memory maintenance. The particular posterior region or regions most activated in working memory roughly coincide with the region(s) containing the most modality-specific memory cells in the monkey. Neuropsychological data implicate those regions in the learning, discrimination, and long-term memory of modality-specific material. Functional imaging, further, supports the conclusion that working memory is based on the sustained activation of a widespread cortical network of long-term memory or cognit. That network unifies neuron assemblies in noncontiguous cortical areas and represents the associated aspects of the memorandum, executive as well as sensory, including the ad hoc trial- or situation-specific information.
HOW ARE MEMORY NETWORKS MADE AND ORGANIZED?
At the foundation of all learning and memory, there are certain changes in the membrane of nerve cells that are common to all organisms (Kandel, 2000). These changes, largely mediated by synaptic modulation, usually take place in the protein structure of the postsynaptic membrane and entail changes in the excitability of the cell. Evidence mostly from invertebrate organisms and from the mammalian hippocampus indicates that the synaptic modulation of neural circuits in learning and memory obeys the principles theoretically formulated in the mid-20th century by Hayek (1952) and Hebb (1949). One such principle is the facilitation of connection between two cells when both fire repeatedly together, one exciting the other. Another principle—emphasized by Hayek—is the facilitation of the response of a cell to two stimuli arriving to the cell simultaneously (synchronous convergence). Arguably, those two principles are reducible to one, especially if recurrent axons are taken into account. One unifying property of synaptic memory-forming mechanisms is embodied in both principles: the temporal coincidence or the near coincidence of synaptic events.
The hippocampus plays a major role in the consolidation of new “declarative” memory (Squire, 1986; Cohen & Squire, 1980), that is, memory accessible to consciousness, which includes autobiographical and semantic memories (Tulving, 1987). It is widely assumed that, in the process of consolidation, the hippocampus cooperates with the neocortex, where long-term memory ultimately settles. On the basis of psychological testing of hippocampal patients, however, the hippocampus has been excluded from so-called procedural or “nondeclarative” memory (Zola-Morgan & Squire, 1993). Two basic problems remain unresolved (Squire & Bayley, 2007; Frankland & Bontempi, 2005). One is to construe in the hippocampus a temporary “map” or depository of the complex information in autobiographical memories, with its mixture of new and old, semantic and episodic, explicit and implicit. The other is to reconcile a memory-consolidating role of the hippocampus with its participation, at least in rodents, in the encoding of spatial locations (Kjelstrup et al., 2008; O'Keefe & Recce, 1993) and olfactory memory (Eichenbaum, Fagan, Mathews, & Cohen, 1988; Staubli, Fraser, Kessler, & Lynch, 1986). In lower mammals, the hippocampus—ancient cortex—possibly plays with regard to vitally adaptive memories (olfaction, spatial navigation, and touch) the same role that the neocortex plays in higher mammals with regard to more elaborate perceptual memories.
Whereas the majority of neuropsychological memory studies of the hippocampus concentrate on memory acquired through the senses (perceptual or declarative), the hippocampus is most probably also involved in the formation of motor or executive memory, that is, the memory of actions. The anatomical connections of the hippocampus with frontal cortex are well developed (Amaral, 1987; Van Hoesen, 1982). One of the first formulations of the synaptic concept of memory applied specifically to executive memory (Cajal, 1923). In the absence of experimental proof, it is reasonable to speculate that the principle of temporal coincidence applies to the formation of motor memory as well as to that of perceptual memory. In the case of motor memory, the coinciding inputs may be either proprioceptive or efferent copies of movement (McCloskey, 1981).
In sum, the hippocampus enables memory formation and consolidation in the neocortex. Here, the newly formed cognits organize themselves (Kohonen, 1984) at various hierarchical levels depending on their complexity or abstraction: from the simplest and most concrete at the bottom, in sensory and motor cortices, to the most complex and abstract at the top, in the higher association cortex of the occipital–temporal–parietal and prefrontal regions. As it is acquired, each new memory or item of knowledge develops from the bottom–up, from the lowest sensory and motor levels to the highest level in cortex of association. That development follows three largely coinciding anatomical gradients: (1) a phylogenetic gradient of increasing cortical volume (Northcutt & Kaas, 1995; Rockel, Hiorns, & Powell, 1980) resulting from evolutionary duplication of areas by genetic mutation (Fukuchi-Shimogori & Grove, 2001; Rakic, 2001); (2) an ontogenetic gradient of increasing myelination and maturation (Barkovich, 1995; Conel, 1963); and (3) a connectivity gradient along either of the two ascending cortical hierarchies, motor and sensory (Petrides & Pandya, 2002; Felleman & Van Essen, 1991; Jones & Powell, 1970). That connectivity is reciprocal at every step, with feed forward as well as feedback.
The theoretical schema of Figure 13 shows the relative position of long-term memories and cognits, after they have been consolidated, in the perceptual and executive cortical hierarchies. The schema is useful as a general organization plan of cortical memory and as a graphic statement of basic principles. Further, it leads to several testable predictions (below). However, it is not intended in any way as “memory map.” Also, memories and knowledge are not neatly stacked in their hierarchies as the figure suggests. Indeed, most memories are to some degree heterarchical; that is, they contain network components at several hierarchical levels.
All individual memory derives from what I call phyletic memory, in other words, from the “memory of the species,” the structural phenotype of sensory and motor systems at birth. The primary sensory and motor cortex can be rightfully called phyletic memory for the following reasons: (a) it is a form of structural “memory,” acquired in evolution, of the sensory and motor means to adapt to the environment; (b) this primal memory, the cortical sensory–motor apparatus, has to be used, “rehearsed,” in certain critical periods after birth before individual memory can be effectively formed over it; (c) thereafter, phyletic memory is “recalled,” voluntarily or involuntarily, in every sensation or motor action of the adult organism; and (d), like other memories, phyletic memory is to some degree plastic and recoverable after injury.
The upward fanning of memory networks in the scheme of Figure 13 emphasizes the divergence of functional connectivity from phyletic memory to higher, more abstract and complex memories. As they consolidate, the latter memory networks become widely distributed (the opposite of the proverbial “grandmother cell”). That organizational feature would also agree with the principle that the larger networks (e.g., conceptual, semantic) are formed to a large extent by the repeated coactivation and instantiation of similar, more concrete (e.g., episodic) memories. Upper cognits would thus derive and generalize from lower cognits; the latter nested within the former.
At all levels, the two hierarchies of networks are interconnected by fibers that reciprocally link the cortex of the frontal lobe with that of posterior regions. These fibers establish the mutual associations between perceptual and action networks. In behavior, language, and reasoning, they support the active engagement of those networks in such operations as working memory and perception–action cycle.
In summary, the structure of long-term memory in the present network memory paradigm has the following major features: (1) it is hierarchical but compatible with a degree of heterarchical organization and dynamics; (2) it contains perceptual cognits mainly in posterior cortex, executive cognits mainly in frontal cortex; and (3) perceptual and executive cortices—and their cognits—are joined by long reentrant and reciprocal fibers, which serve working memory and the dynamics of the perception–action cycle.
The postulated functional architecture of memory networks leads to the following predictions. Some of them have already been partially tested and used to support the argument for the new paradigm. They need, however, expanded testing to confirm or reject this paradigm.
A. Cortical lesions will induce memory deficits depending on the location and extent of the lesion. In the posterior hierarchy, from sensory cortex to association cortex, lesions will affect the formation, retrieval, and working memory of progressively higher perceptual content. Small lesions at low level (sensory cortex) will affect simple sensory cognits. Larger lesions at higher levels (temporal, parietal cortex) will affect larger, more complex cognits (unimodal and polymodal agnosias, aphasias, and amnesias). Conversely, in the executive hierarchy, from motor to pFC, lesions will affect progressively higher executive content. Small lesions of motor cortex will affect the representation of simple movements by discrete muscle groups. Larger lesions of premotor cortex will affect the representation of movements defined by goal and trajectory. Still larger lesions in the pFC will affect the highest, most complex executive memories and knowledge, including rules and plans.
B. On the assumption of partial commonality of anatomical substrate for long-term and working memory, it can be predicted that the activation of cognits in working memory will elicit electrical and functional imaging signals from the cortical areas representing the memorandum. In those areas, during working memory, microelectrodes will record persistent unit discharge and synchronous high-frequency LFP oscillations. Sustained working memory will elicit imaging signal from those areas. By manipulating the category and the context of the memorandum, it will be possible to vary the source and location of the signals and thus the spread and location of the activated cognits. For example, a concrete sensory memorandum will activate a relatively small region of sensory association cortex, such as the superior (auditory) or the inferior (visual) temporal gyrus. A more complex stimulus with associations of more than one sensory modality will activate multiple sensory association cortices. In all working-memory tasks, the activation will be interregional, involving simultaneously prefrontal and posterior cortex, as the activated cognits will encompass perceptual as well as executive networks.
BEHAVIORAL NETWORK DYNAMICS: THE PERCEPTION–ACTION CYCLE
In the cortex, as in the rest of the brain, there are no “systems of memory,” but there is the memory of systems. All cortical systems have their own memory, which is inextricable from the operations they perform. The substrate for process is inseparable from the substrate for representation. Cognitive networks contribute to behavior by performing the sensory and the motor functions they represent.
From evidence reviewed, it can be reasonably inferred that, in goal-directed behavior, posterior and frontal cognits join together to coordinate the action. At high levels of the cortical hierarchies, prefrontal networks, which represent broader actions and longer term executive goals, successively activate subjacent networks that represent shorter term, intermediate actions and goals. At every step, action is guided by feedback from previous actions. The entire sequence, with its subordinate steps, is generated and carried out continually by executive networks at various levels, integrating stimuli from the environment (internal and external) with feedback signals from that environment, all within the framework of the perception–action cycle.
The perception–action cycle is a basic biological principle that governs the functional relationships of the organism with its environment. As a process, it is the cybernetic circle of sensing and acting that guides the organism to its goals. The concept originated in biology (Uexküll, 1926) and eventually entered neurology (Weizsäcker, 1950), cognitive science (Neisser, 1976), and computational neuroscience (Arbib, 1985). The perception–action cycle operates at all levels of the nervous system, from the spinal cord to the cerebral cortex. In the course of complex behavior, it engages neural networks at every hierarchical level of the neocortex, following processing paths that course through the environment and through connections between cortical areas (Figure 14). Action may be initiated anywhere in the cycle, in the internal or external environment. Once the cycle is engaged, its networks become engaged in series as well as in parallel, with the qualification that the interactions between networks may link different levels heterarchically. Another qualification is that the cycle is at all levels bidirectional: feed forward is accompanied by internal feedback. That feedback serves as a kind of corollary discharge (Teuber, 1972) to prepare for impending perception as well as action.
Highly automated, overlearned, or instinctual behaviors and habits need not engage the cognits of the cerebral cortex. They can be sequentially performed in chainlike fashion through shunts at lower levels of the cycle. The cortex becomes engaged in the cycle, however, when there are discontinuities in the sequence, especially if the latter requires temporal integration in the face of uncertainty or ambiguity—as in working-memory tasks. Internal feedback then serves perceptual as well as executive attention.
Thus, in the course of a demanding temporal gestalt of behavior, language, or reasoning toward a goal, the perception–action cycle orderly recruits a series of memory networks, each modulated by internal feedback. With the methods available, the order can be traced only coarsely. A promising analytical method is the computing of Granger causality on electrical signals. Essentially, this method allows the study of information flow by analysis of multivariate changes in the interdependence of time series from several sources (Blinowska, Kus, & Kaminski, 2004; Brovelli et al., 2004). The application of an algorithmic transfer function to the trains of electrical events makes it is possible to predict the directionality of the events between sources and thus to infer the causal relationships between them. Granger causality analysis has been successfully used in tracing electrocortical activity in networks of frontal and parietal areas controlling hand movement in a visuomotor discrimination task (Brovelli et al., 2004).
In neuroimaging, time-series analysis and correlation methods provide further evidence of the spatial and dynamic characteristics of large-scale cognitive networks. Bullmore et al. (1996) provided evidence of the activation of vast networks during a task that requires visual and semantic processing. Those networks show major foci of activation in extrastriate cortex, angular gyrus, superior and middle temporal gyri, premotor cortex, and Broca's area. Significantly, there are large functional distances and discontinuities (negative connectivity) between some of the foci. In other studies (Newman, Just, & Carpenter, 2002; Lowe, Dzemidzic, Lurito, Mathews, & Phillips, 2000), correlation and synchronization gradients are detected between the components of a large network in working memory. As the collaboration of frontal and parietal regions increases, the correlation and the synchronization between them also increase.
Executive frontal networks are activated during the mental planning of serial movement (Baker et al., 1996; Roland, Larsen, Lassen, & Skinhøj, 1980; Ingvar & Philipson, 1977). Those networks, therefore, seem to be the depositories of so-called “memory of the future” (Ingvar, 1985). Fulfilling Jackson's prediction with regard to anterior frontal areas (Jackson, 1882), those same networks are involved in the coordination of the planned action, beginning with the evocation of the objects leading to that action. For example, whereas the viewing of animals activates posterior areas, that of tools (“action objects”) activates premotor cortex (Martin, Wiggs, Ungerleider, & Haxby, 1996). Further, those networks are involved in the actual implementation of the actions, as in the performance of the Tower of London, a test of planning ability (Morris, Ahmed, Syed, & Toone, 1993).
Just as there is an orderly representation of actions in frontal cortex, from the most abstract and complex in pFC to the most concrete in motor cortex, the orderly execution of actions follows that trend. Thus, the processing in the executive memory side of the perception–action cycle descends from the concept and plan of action in prefrontal networks downward to specific action networks in motor cortex. Especially persuasive in this respect are the studies by Badre and D'Esposito (2007) and Koechlin, Ody, and Kouneiher (2003). Both use behavioral paradigms in which the motor response to a sensory stimulus is contingent on progressively more remote associations of the stimulus or the response.
In the 2003 study (Figure 15), responses depend (a) on a simple feature (color), (b) on the presence of an additional feature (pattern) that provides the “context,” or (c) on a recent instructional—visual—cue, a prior contingency that the authors call “episode.” Thus, from conditions a to c, the response to the stimulus is determined by information of increasing complexity and associative load. In condition c, the subject must also integrate information across time. Condition a activates premotor cortex; condition b, in addition, activates posterior pFC; and condition c, in addition, activates anterior pFC. Further, in the third task, the path coefficients of activation suggest a processing “cascade” that originates in anterior pFC and courses through premotor to motor cortex.
In sum, the activation of sensorimotor cognits progresses down the executive hierarchy. Two implicit qualifications seem necessary, however, to properly interpret the results of those studies within the network paradigm. One is that, according to the anatomy, the connectivity is reciprocal between all stages of the executive hierarchy. The other is that the processing toward action is not only serial, as most hierarchical models imply, but also parallel.
At the end of the previous section, a set of empirical predictions has been presented that mainly derives from the structural characteristics of the network paradigm discussed in this review. I finish this section with another set of predictions deriving mainly from its dynamic characteristics:
A. Serial behaviors that require sequential decisions informed by the environmental consequences of the subject's actions will reveal the temporal alternation of Granger causality between frontal and posterior areas. This alternation will be revealed not only by LFP records but also by neuroimaging records. Because the necessary behavioral and recording methods are best suited to the nonhuman primate, those hypotheses should be tested in monkeys. Neuroimaging records will suitably be obtained by near-infrared spectroscopy (NIRS), an optical-imaging method with less spatial resolution but considerably greater temporal resolution than fMRI (Fuster et al., 2005).
B. Working memory will elicit sustained, widespread—interregional—activation of frontal and posterior cortex, and this will be manifest in multiple-unit, EEG, LFP, and NIRS signals. On the assumption that the activation reflects reentrant excitation within and between cognits, regions presumed to contain networks representing the memorandum and the operant response will show high degrees of covariance and coherence—especially in the high-frequency range.
C. The reversible—for example, cryogenic—lesion of frontal cortex will induce a deactivation of posterior cognits and a deficit in working memory. These will be manifest in multiple-unit and LFP records of posterior cortex and in a concomitant behavioral memory deficit. These manifestations of deficit will result from a dual reversible effect of the procedure: (a) interruption of the perception–action cycle at the top and (b) interruption of reentry loops between frontal and posterior cortex maintaining active memory.
Conclusions and Future Research
A large body of recent evidence endorses a new network paradigm of the structure and dynamics of memory in the cerebral cortex. That evidence suffices to establish the paradigm's basic principles, which sharply distinguish it from other models of cortical memory. According to it, memories as well as items of knowledge consist of distributed and hierarchically organized cortical networks. These memory networks or cognits consist of dispersed neuron populations and the associative connections that link them. Those connections, commonly bridging noncontiguous areas of the cortex, are formed, enhanced, and expanded by experience-dependent synaptic modulation. Cognits overlap and interconnect extensively within and between hierarchical levels (heterarchically). Thus, a neuron or a neuron population can be part of multiple memories or items of knowledge.
Whereas the paradigm is based on vast empirical evidence, it is difficult to conceive of one single critical experiment to prove it correct. Definitive proof would require the simultaneous application to the entire cortex of analytical methods with multiple scales of spatial and temporal resolution, which is now impractical. In the absence of such capability, we can assume that the model is essentially correct and attempt to strengthen experimentally the principles outlined in this review. Nonetheless, one of the merits of the paradigm is that even with current methods, those principles can be proven wrong. Indeed, the paradigm would be falsified if it were shown that (1) specific memories or items or knowledge are exclusively represented in discrete domains of cortex and not endowed with experience-dependent plasticity; (2) memory or knowledge representations are not interregional and are deprived of hierarchical organization and nesting; (3) anatomical relationships between cell groups are immaterial to the memory code; or (4) substantial numbers of cells or cell groups in separate cortical locations are not attuned to the associated constituent properties of a memory or an item of knowledge.
More research is needed for complete description of the emerging paradigm in physiological and computational terms. To that end, a number of predictions—advanced in the previous two sections—need testing. Because of the model's distributed, associative, and integrative architecture, special efforts must be made to explore its constituent networks with a variety of different methods. Especially useful will be the simultaneous recording of electrical and imaging signals from multiple sites in memory tasks. Unit discharge, LFPs, surface EEG, and NIRS should be recorded from monkeys performing working-memory tasks. This combination of signal-recording methods, together with advanced computational methods, should allow the exploration of active memory networks with varying scales and resolution. Signals of different origin should be submitted to computational analysis in multiple spatial and temporal scales, with emphasis on coherence, covariance, and Granger causality. As heretofore, working-memory tasks, with their time-bracketed active memory, should continue to provide the ideal setting for this research.
Finally, a pressing issue is the accessibility of long-term memory, in the active state, to consciousness. The available evidence indicates that conscious cognition is a graded function, largely dependent on synaptic strength and degree of cognit activation. These inferences open new avenues for understanding priming and preconscious phenomena. Moreover, research on these issues may lead to new procedures for memory enhancement as well as the rehabilitation of old associative paths or the opening of new ones.
I wish to thank Thomas Albright, Mark D'Esposito, and Steven Bressler for their valuable comments on earlier versions of the manuscript.
Reprint requests should be sent to Joaquín M. Fuster, UCLA Semel Institute, 740 Westwood Plaza, Los Angeles, CA 90095, or via e-mail: firstname.lastname@example.org.