Dynamics Are the Only Constant in Working Memory

The central goal of working memory research is to understand how we temporarily hold information in mind while moving through the world to achieve our behavioral goals. Working memory is a critical cognitive function that allows us to link together our experiences into a coherent narrative. Not surprisingly then, in the talks and classroom lectures that we as scientists give on working memory, we all love to begin with engaging real-world examples. We might show a crowded supermarket aisle filled with colorful vegetables and highlight how you use working memory to hold your grocery list in mind while searching for the perfect strawberries. Or, we might show a busy city street and explain how you can find a friend in the crowd by visualizing their face or signature pink beret. There is no shortage of vivid examples for opening talk slides. However, around Slide 5, we invariably pivot to the following: a few discrete gray boxes representing a computer screen, set sequentially on a timeline. In one of the first boxes, there is a swipe of stripes or splash of color, which people are asked to remember. Then, there is a gray box, meant to indicate the working memory delay. And in the last box, there are again some stripes or colors used as a test.

Those of us engaged in working memory research are so familiar with this pivot from real world to laboratory that we scarcely notice it. First-time listeners, however, might struggle to see the connection. Of course, there is a reason we all love our artificial tasks, and there is great value in them. As psychologists, we learn that a task is a means for exerting experimental control at the cost of naturalism. As neuroscientists, we find that artificial tasks are particularly useful for shepherding people's brains into approximately the same state over and over again so that we can extract the signal from the noise. By contrast, if you find yourself wandering through the supermarket aisle one fine morning as per the example on Slide 1, it is difficult (currently, near impossible) for a neuroscientist to glean information from your mind.

Because they are simple, we like to think of our typical working memory tasks as predictable and interchangeable. Like following a recipe in the kitchen, you can predict behavior based on the time allotted for encoding and retention. Like parts from IKEA shelves, you can mix and match task components to get the desired effect. However, ongoing work has shown how even the simplest task components are not so formulaic. For example, one long-standing question that contributed to the split of cognitive psychology from behaviorism is how one stimulus can map onto many different behaviors. Drawing an ace from the deck is sometimes the best card and sometimes the worst—it all depends on which card game you are currently playing. In technical terms, the same stimulus triggers different mental operations and behaviors in different contexts.

Understanding how one stimulus can be flexibly mapped to different behaviors is a particularly challenging problem when viewed from the perspective of individual neurons. In visual neuroscience, it is often fruitful to characterize neurons' tuning preferences. It is easy to imagine mental representations arising from stably tuned neurons—if you want to represent a “vertical” item, in theory, you could achieve this by having vertical-preferring neurons persistently fire to bridge a delay. Yet, a scheme like this cannot fully account for working memory's flexibility: Sometimes, “vertical” could mean “press button A,” and other times, it could mean “look to your right.” To rapidly link arbitrary pieces of information together requires flexible shifts in the representation of information. A key insight from Stokes et al. (2013) is that a multidimensional landscape emerges when an individual neurons' activity is viewed in relation to the activity of all other neurons. In this landscape, each neuron traverses a single dimension over time, and all neurons together traverse a highly dynamic trajectory that can settle into stable states during various epochs of the working memory task. This dynamic and multidimensional state space can be considered across all neurons, but can also be condensed back into fewer dimensions by looking only at those components that explain most of the variance in a given task (using a dimensionality reduction technique like PCA). From Stokes et al. (2013), we learn that a lower-dimensional stable activation state can be observed during working memory maintenance, which reflects the temporarily configured network state that is dynamically tuned according to task goals. For example, a stable state can map how a memorized stimulus relates to an appropriate decision required during response. Critically, by taking into account the multidimensional nature of neural codes, many flexible behaviors can suddenly fit rather effortlessly into our theories about working memory.

Work on multidimensional codes in the context of mapping one stimulus to multiple behaviors has shown how low dimensional states can be flexibly assembled and reassembled to adapt to moment-to-moment behavioral demands. Even more remarkably, subsequent work has revealed that activity across large populations of neurons can remain highly dynamic even when the stimulus and task demands are held constant (Wolff, Jochim, Akyürek, Buschman, & Stokes, 2020; Murray et al., 2017; Spaak, Watanabe, Funahashi, & Stokes, 2017). In a visual working memory task, observers were asked to remember a simple stimulus (like an oriented grating or spatial location). The overall pattern of neural activity during the memory delay is volleyed through a series of changes over time. Despite these rapid temporal dynamics in the population at large, the coding-scheme, or the low-dimensional subspace that represents the simple stimulus, remained remarkably stable, exhibiting only small drifts over time (Wolff et al., 2020; Murray et al., 2017). From our conscious perspective, memory of a simple stimulus such as an orientation is like a statue held “fixed” in our mind's eye. From a neural information processing perspective, it is like a river finding its way down the different grooves in a landscape—all the while keeping the memories afloat on a stable boat (Panichello & Buschman, 2021; Panichello, DePasquale, Pillow, & Buschman, 2019).

Understanding working memory codes as highly dynamic and evolving across time was a transformative idea from Stokes and colleagues (2013), and we are only slowly beginning to understand more about the ways in which memories are maintained from this novel perspective. For example, recent work has adopted the dynamic coding framework developed by Stokes to address one of the classic questions in philosophy, cognitive psychology, and neuroscience: When you hold a memory in mind, how do you know that it is a memory, and not a representation of incoming sensory information? Put another way, how does your brain attenuate interference between internal thoughts and sensory information? By examining the state space of multi-unit recordings, Libby and Buschman (2021) demonstrated that the sensory tuning of some neurons is stable during the maintenance of information in memory, whereas the tuning of other neurons is inverted with respect to sensory tuning. The net result is a rotation of the state-space representation of the memory code with respect to the sensory code, providing a mechanism to separate memory representations from sensory representations. Although this finding suggests a means of mitigating interference between memories and sensory inputs, these dynamics complicate the process of decoding the remembered information to guide behavior. How can a specific remembered feature be “read-out” when that feature is no longer in its original sensory-like format? Several studies—inspired again by Stokes' approach to dynamic codes—have shown that neural response patterns can be highly dynamic over time, all while preserving the structural relationship between the stimuli being remembered, so that they remain separable in a stable subspace (e.g., Wolff et al., 2020; Bouchacourt & Buschman, 2019; Murray et al., 2017; Spaak et al., 2017).

The inspiration sparked by the idea of dynamic codes (although employing different analysis approaches) has steadily trickled into the neuroscientific thinking on an equally long-standing question: Where is the cortical locus of mnemonic representations? The classic story is that sustained spiking activity in pFC is the key mechanism supporting the stable memory representations guiding behavior. However, Stokes' demonstration of dynamic codes forced the field to reconsider. With the notion of multidimensional and dynamic subspaces as a starting point, should we even expect any single neural locus (e.g., pFC), or a single mechanism (e.g., sustained spiking), to be the seat of working memory? Instead, for any given working memory task—be it remembering colored squares or remembering your grocery list—there should be some distributed and temporally evolving pattern of neural activity that flexibly recruits the brain areas and neural mechanisms needed to get the job done (Courtney, 2022; Iamshchinina, Christophel, Gayet, & Rademaker, 2021; Lorenc & Sreenivasan, 2021; Christophel, Klink, Spitzer, Roelfsema, & Haynes, 2017; Sreenivasan, Vytlacil, & D'Esposito, 2014). Indeed, recent work has shown that information about visual stimuli can be recoded into motor representations if a response is known in advance (Henderson, Rademaker, & Serences, 2022; Boettcher, Gresch, Nobre, & van Ede, 2021), or can be recoded into an abstracted mnemonic format (Kwak & Curtis, 2022; Rademaker, Chunharas, & Serences, 2019). In summary, Stokes' work inspires the idea that there is not one place or one mechanism that is a constant during working memory. Instead, the only constant is flexibility and the temporal dynamics that connect sensory inputs to context-specific behavioral goals.

One implication of the dynamic coding framework is that there is not a general solution to the “problem” of working memory. To understand working memory, we need to reckon directly with its immense flexibility. To do so will require both devising new tasks, as well as carefully considering how changing mental states and behavioral contexts impact processing in even the simplest of tasks. This does not mean that some principles of working memory will not generalize—some “solutions” might be more or less similar given the relationship between different contexts. However, a better understanding of the dynamics of working memory—as revealed by Stokes and others in the past decade—should motivate more consideration about the design and relevance of our tasks for everyday life, and help to kick us out of the attractor-like state of thinking that there is only one way to implement working memory in the brain.

Reprint requests should be sent to Kirsten C. S. Adam, Department of Psychology, University of California, San Diego, La Jolla, CA 92093-0109, or via e-mail: [email protected].

Kirsten C. S. Adam: Conceptualization; Writing—Original draft. Rosanne L. Rademaker: Conceptualization; Writing—Original draft. John T. Serences: Conceptualization; Writing—Original draft.

Kirsten C. S. Adam, National Science Foundation (https://dx.doi.org/10.13039/100000001), grant number: 2104630. John T. Serences, National Eye Institute (https://dx.doi.org/10.13039/100000053), grant number: R01 EY025872.