Executive control refers to the regulation of cognition and behavior by mental processes and is a hallmark of higher cognition. Most approaches to understanding its mechanisms begin with the assumption that our brains have anatomically segregated and functionally specialized control modules. The modular approach is intuitive: Control is conceptually distinct from basic mental processing, so an organization that reifies that distinction makes sense. An alternative approach sees executive control as self-organizing principles of a distributed organization. In distributed systems, control and controlled processes are colocalized within large numbers of dispersed computational agents. Control then is often an emergent consequence of simple rules governing the interaction between agents. Because these systems are unfamiliar and unintuitive, here we review several well-understood examples of distributed control systems, group living insects and social animals, and emphasize their parallels with neural systems. We then reexamine the cognitive neuroscience literature on executive control for evidence that its neural control systems may be distributed.
Executive control refers to the brain's ability to regulate its own processing. It coordinates multiple competing demands, controls attention, gates working memory, and encodes and retrieves long-term memories. It also maintains and switches task set, inhibits disadvantageous actions, and regulates the explore/exploit tradeoff and curiosity (Kidd & Hayden, 2015; Braver & Barch, 2006; Miller & Cohen, 2001; Shiffrin & Schneider, 1977). Understanding executive control is critical for understanding self-control and its failures (Aron, Robbins, & Poldrack, 2014; Hare, Camerer, & Rangel, 2009; Knoch & Fehr, 2007). More broadly, failures of executive control are hallmarks of many diseases, including addiction, depression, and obsessive-compulsive disorder, and successful treatments of these diseases often target executive control (e.g., Milad & Rauch, 2012; Kalivas & Volkow, 2005).
A brain can be understood as a control system, a collection of interacting components within an organizational structure that produces adaptive actions based on information about the current state of the internal and external worlds (Pezzulo & Cisek, 2016; Gallistel, 2013). As we process sensory inputs and generate actions, the brain monitors that processing, and if it detects the need to change, it regulates it. But how is executive control in the brain implemented by the interactions of its constituent parts, individual neurons?
Modular and Distributed Control Systems
The standard approach to understanding control starts with the assumption of modularity. In a modular control system, regulation is derived from a central controller, which is a discrete subsystem with a specialized function. Applied to the brain, it is theoretically possible to draw a line through anatomical space separating localized control regions or circuits (often the pFC and striatum) from more basic processing (caudal cortical) regions (Botvinick, Braver, Barch, Carter, & Cohen, 2001; Miller & Cohen, 2001). This specialization means that control regions (or networks) regulate, but do not participate in, the underlying stimulus-to-action transformation processes (Figure 1). Such a view is consistent with a long tradition emphasizing the brain's modular architecture (Kanwisher, McDermott, & Chun, 1997; Minsky, 1988; Fodor, 1983). But it is not the only possible view.
The alternative approach envisions executive control as distributed processes in which there is no dedicated and specialized controller (Figure 1). Instead regulatory functions are dispersed across a large number of individual elements or carried out by the interaction among them (Couzin, 2009; Sumpter, 2006). In distributed systems, control elements are colocalized with processing elements, and those elements have somewhat autonomous function. For this reason, they are often called “agents.” These agents (or any other individual elements in a distributed control system) sense the properties of their local environment and adjust their own behavior based on simple rules. Agents normally have no knowledge of the overall state of the system, and the response of the system as a whole is often qualitatively dissimilar from those of the elements. In other words, in such systems, control is often an emergent function (McClelland et al., 2010; Mitchell, 2009; Hofstadter, 1985).
The distributed viewpoint derives inspiration from early studies on cybernetic, connectionist, and PDP models (Rumelhart, Hinton & McClelland, 1988; Hopfield, 1982; Weiner, 1948). As noted in a review of the topic by Botvinick and Cohen (2014), the connectionist heyday of the late 1970s and early 1980s coincided with the development of formal ideas of control (Norman & Shallice, 1986; Shiffrin & Schneider, 1977; Posner & Snyder, 1975). It is ironic then that almost all models of executive control, even relevant PDP models, take as given that control is functionally and anatomically modular (Botvinick & Cohen, 2014). Nonetheless, history has vindicated this approach: The modular idea is well supported by empirical data. Specifically, neuroscientific research consistently points to dorsal prefrontal structures (especially the dorsal ACC [dACC], see below), as well as superior parietal cortex and parts of the brainstem as the brain's control system (Sleezer & Hayden, 2016; Botvinick & Cohen, 2014; Shenhav, Botvinick, & Cohen, 2013).
Revisiting the Distributed Processing View
Still, we believe that it is time to revisit a distributed approach to control. Several factors motivate this belief. First, our understanding of the neuronal (i.e., single unit) responses of the putative executive regions is only now maturating. Some of this work emphasizes the broad overlap in functions of the prefrontal and posterior regions; these functions appear to include both processing and executive roles (Sleezer & Hayden, 2016; Cisek & Kalaska, 2010; Postle, 2006; Awh & Jonides, 2001). Second, new anatomical and functional techniques emphasize the fundamentally nonmodular organization of the brain (Wang & Kennedy, 2016; Kristan & Shaw, 1997; Farah, 1994). Third, major recent advances in computation have come from abandoning classic symbol manipulating systems in favor of deep learning algorithms that are distributed and recurrent (e.g., LeCun, Bengio, & Hinton, 2015). These approaches highlight the power and flexibility of nonmodular network organizations. Finally, recent years have seen a greater understanding of the mechanisms of distributed control in nonbrain biological systems, leading to a greater appreciation of the strengths and of the biological plausibility of such systems (Couzin, 2009; Passino, Seeley, & Visscher, 2007).
Reified models of executive control—in which conceptual elements like monitor, controller, and processor have direct correspondence with neuroanatomy—are intuitive. But distributed models are less so. To mitigate this problem here, we offer a summary of the basic principles of distributed control systems, with an emphasis on natural examples.
PRINCIPLES OF DISTRIBUTED CONTROL SYSTEMS
Principle 1: Horizontal Information Flow
Within a modular control system, information flows linearly from lower level processing units to the controller. By contrast, information flow within distributed systems is characterized by horizontal communication between adjacent members. In other words, information is derived from neighbors, not from a central communicator. Consequently, no single member of a distributed system is knowledgeable about the entire system. Each member can know what their neighbor is doing and possibly what their neighbor knows through localized interactions.
A good example of information flow within a distributed system is a herd of baboons on the move (Papio anubis; Strandburg-Peshkin, Farine, Couzin, & Crofoot, 2015; Couzin & Krause, 2003). Even though they have a hierarchical dominance system, no single member of the troop knows for sure where to go, but several members have some limited and likely noisy knowledge—as in many collectively moving animals (Codling, Pitchford, & Simpson, 2007; Bergman & Donner, 1964). Individuals begin to head off toward their best guess, and as they do this, troop members compute the average of the members they observe (Figure 2). Unlike in a modular system, each member may be simultaneously a decision-maker and a data point for other decision-makers.
Normally this strategy leads efficiently to a rapid consensus (Couzin & Krause, 2003). In cases where there are two different modal preferences—say, when northeast and northwest are both good directions but true north is not, this averaging strategy leads to a suboptimal choice (Figure 2B). For this reason individuals should be—and are—sensitive to bimodal distributions among the group and, in that case, randomly choose one of the two modal directions (Strandburg-Peshkin et al., 2015).
In this example, the input is the environmental clues (including memories) about the best direction to head, and the output is a group path. Information is distributed across individual troop members who communicate locally with each other. Drawing from the local interactions among members, the group chooses a better output than all the constituent individuals. The decision is also controlled in a closed-loop manner: The group can monitor its own performance (it can detect split voting) and regulate its voting strategy (averaging when there is one best guess and bifurcating when there are two equally valid directions of travel), even though no individual serves as the specialized monitor or regulator.
The idea of horizontal flow of information from adjacent members is also often a description of neuroanatomical organization. Neurons, like troop members, tend to have limited view of the activity of the whole, limited ability to communicate with the whole, incomplete information, no knowledge of the larger factors that determine the group's well-being, and no obvious leadership. However, neurons do have a rich network of connections to adjacent neighbors and cortical areas that support a localized flow of information. Although the brain also has centralized global signaling, in the form of neuromodulators and cortical oscillations, the bandwidth of these signals is limited, and the timing may be too slow to affect online decision processes. Similar to a baboon troop, the information gained from equal and adjacent members has a large effect on the regulation of its neural function.
Principle 2: Stigmergy
In the case of the baboons, it is notable that the control signal is the movement of neighbors. Thus, in a strongly nonmodular way, the control signal is precisely the output of the underlying process (also movement of individuals). It is a stigmergic system (Couzin, 2009; Theraulaz & Bonabeau, 1999). A familiar example of stigmergic signaling is lawn shortcut generation on college campuses. A student following the trod path also—weakly but surely mdash;strengthens it (Figure 3). Another example is pheromonal trails in foraging ants (Wilson, 1971). As a scout forages, she lays a scent that other scouts will follow to valuable food sources. The scent evaporates quickly, so rich food patches, which attract many ants, will have stronger paths leading to them. An ant that, by chance, discovers a shortcut will produce a trail with a stronger scent (because, being shorter, it takes less time to traverse and thus has more scent; Beckers, Deneubourg, & Goss, 1992). In this way, pheromones allow ant colonies to find rich food sources and develop shortest path routes without any centralized control (Jackson & Châline, 2007).
Another example of stigmergic control comes from the process of neural differentiation of sensory organ precursors (SOPs) within the developing fly brain (Drosphilia melanogaster; Navlakha & Bar-Joseph, 2011). During development, some cells within the neural clusters of the fly brain become SOPs. Determination of which cells become SOPs follows an algorithmic process wherein undifferentiated cells propose themselves as a possible SOP. If any neighboring cell has already become an SOP, the proposing cell will not differentiate (Afek et al., 2011; Navlakha & Bar-Joseph, 2011). By using information about the structure of neighboring cells, each cell is able to differentiate appropriately so that the whole brain achieves an equal spacing of SOPs. The brain cells do this rapidly and without the need for a monitor or knowledgeable controller sending distinct control signals. All the monitoring and control that is needed occurs locally, within each cell.
Principles of stigmergy within executive control processes relate to neural function quite directly. Neurons produce chemical outputs that modulate responses of downstream neurons. These outputs are both the computational outputs of the neurons and a way to modulate activity of their neighbors. In the short term, excitatory and inhibitory outputs increase and reduce, respectively, the likelihood that the target will fire. In the long term, activity (especially coincident activity) promotes synaptic plasticity thus up- or down-regulating that target's firing on longer timescales. Within cortical regions, these localized interactions could very well lead to emergent control signals without the need for a dedicated controller (Couzin, 2009). Additionally, they share many similarities with Hebbian learning rules. Just as an ant trail is more likely to be followed by another ant, the neural activation of one cell by another strengthens its efficacy of activation for that cell.
Principle 3: Feedback Loops
Feedback is a powerful tool in any dynamical system. It can have positive effects. When fish school, a few peripheral individuals may detect a potential predator and turn away from it (Couzin & Krause, 2003; Treherne & Foster, 1981). Neighbors who follow an average direction rule then turn and also affect their neighbors, the effect multiplies, and the traveling wave of turning fish turns the whole school away. The amplification protects many more fish than were able to detect the predator. Similarly, feedback loops are a mainstay of other distributed leaderless systems.
However, feedback loops can be dangerous as well (Giraldeau, Valone, & Templeton, 2002). One important example of a feedback loop is a marketplace bubble (Porter & Smith, 1994; Smith, Suchanek, & Williams, 1988). If a speculator believes a commodity will go up in price, she may bid a slightly greater price than the current one. This bidding will serve as a signal to other investors that the commodity may be a wise investment. As they bid up the price, their initial assessment will be proven to be right, and other investors will gain interest. This pattern can lead to runaway prices, but only up to a point; as soon as this point is reached, the price will crash.
The tendency to boom and bust can lead to market instability and to underinvestment. In marketplaces, centralized control (such as trading limits) can solve these problems. Without that kind of control, avoiding these kinds of malign feedback loops requires careful calibration of the rules each individual follows. Such calibrations often involve complementary negative feedback loops (Grünbaum, 1998). The analogy to brains, which have many overlapping positive and negative feedback loops, is quite direct.
Principle 4: Self-organization through Simple Rules
Many distributed control systems are self-organized (Sumpter, 2006). Classic examples of self-organization include bird flocks and fish schools. No leader bird rallies its mates and tells them where to fly; nor does a leader monitor the flock and guide its performance like coxswain on a crew team. Instead, the structure of the bird flock is a consequence of several simple principles followed by all individuals. These include rules about distance between adjacent birds (not too far and not too close, more or less) and rules about when to turn (follow the group average; Couzin & Krause, 2003). The specific rules, not a leader-bird, determine the shape of the flock (Figure 4).
Self-organization is an appealing principle because it is easy to implement and is robust to degradation (Sumpter, 2006). In contrast, the centralized systems are vulnerable to the loss of the controller: Remove a switch and the whole railyard breaks down; remove the coxswain and the rowers start hitting each other's oars; remove one bird and the flock swiftly adjusts. Self-organization also allows complex adaptive behavior without programming expensive control systems. Self-organized systems can be “fast, cheap, and out of control” (Brooks & Flynn, 1989). These features—ease of implementation, graceful degradation, and robustness—make it appealing for analyzing neural systems. One well-known example of a self-organizing system in neurons is central pattern generators, in which the activity of the ensemble is an emergent product of the interactions of the elements, none of which follows the pattern in miniature.
Presumably, we can classify control systems on a spectrum from fully distributed and leaderless to strictly segregated and hierarchical; the specific organization observed for any system will depend on the environment in which it evolved. This fact is important to remember when considering neural systems, which may have some specialization of function.
Principle 5: Quorum Sensing
Agents in distributed systems have very limited field of view in their monitoring capabilities. In other words, it is often difficult to see the forest for the trees. But sometimes it is critical to see the forest to make the best decision. In these cases, agents must engage in “quorum sensing”: a type of consensus-based control mechanism wherein a set threshold or quorum determines the course of action (Mitchell, 2009).
There are many mechanisms for quorum sensing; these can include chemical, hormonal, group members present, and others, what unites them is that they do not require centralized control. Often, individuals can sense the state of conspecifics in their local environment and extrapolate to an estimate of group state. Simply averaging the states of neighbors can be helpful in some circumstances, as in bird flocks and some fish schools. One study showed that an individual schooling three-spine stickleback fish (Gasterosteus aculeatus) can adopt a nonlinear monitoring function that produces better group behavior emergently (Ward, Sumpter, Couzin, Hart, & Krause, 2008). Specifically, groups of fish tended to ignore information from single neighbor but responded when two fish conveyed the same information. This nonlinear criterion can reduce the probability of amplifying noise but can still effectively detect signals.
The need for agents to sense the properties of the whole, or of large subgroups, is a major problem in brain systems as a whole. This problem is acute in executive control systems, which often rely on changing processing as a function of global conditions. Without holistic integrating neurons, it is difficult to imagine a direct solution to the problem. For this reason, studies of quorum sensing systems, which solve the problem indirectly, are particularly likely to be helpful in understanding the neural basis of control.
DISTRIBUTED SOLUTIONS TO CLASSIC EXECUTIVE CONTROL PROBLEMS
Among the important executive control problems are regulation of stop/go behavior, speed–accuracy tradeoffs, and conflict detection and resolution (Aron et al., 2014; Bogacz et al., 2009; Miller & Cohen, 2001; Botvinick, Nystrom, Fissell, Carter, & Cohen, 1999). In this section, we investigate how some distributed control systems handle them through the fundamental elements outlined above. Other important executive functions, which we do not consider, include working memory, attention, task set maintenance and switching, regulating the balance of explore versus exploit behavior, and aspects of reinforcement learning. Several of these have likely correlates in distributed control systems as well (Couzin, 2009; Passino et al., 2007; Couzin, Krause, James, Ruxton, & Franks, 2002).
Stopping and Going: Vibrio fischeri Bacteria
Initiation and inhibition of behavior is a simple and important executive function (Jin & Costa, 2010; Schall, 2001). Coordination of these two antagonistic processes can produce both simple responses and complex behaviors. One important requirement is avoiding intermediate responses, so that the system can either fully stop or fully go, without drifting between the two extremes. Our example of stop/go control in a distributed control system comes from the luminous bacterium V. fischeri (Waters & Bassler, 2005; Miller & Bassler, 2001; Nealson & Hastings, 1979). This single-celled organism lives in the light organ of the Hawaiian bobtail squid (Euprymna scolopes) and emits light when the squid hunts at night. The light serves to camouflage the squid that otherwise would be visible in the form of a moonlit silhouette to prey below it (Visick, Foster, Doino, McFall-Ngai, & Ruby, 2000). During the day, the squid hides from potential predators in the dirt and turns its eyes off by extruding most of the bacteria into the surrounding ocean. As the day progresses, the remaining bacteria reproduce rapidly and, by nightfall, have replenished their stock so that there are enough bacteria to serve as an effective camouflage.
The control problem comes from the fact that the bacteria must not luminesce during the day as they are reproducing. Bioluminescence needs to be both inducible and repressible (Nealson & Hastings, 1979). They do this by waiting until there is a quorum of other V. fischeri bacteria in the squid light organ. But how do they know how many others there are? Quorum sensing. V. fischeri release a chemical known as acyl-homoserine lactone (AHL). They then measure the concentration of this chemical in their local environment by the transcription activator protein LuxR, which creates a complex that induces transcription of genes needed for luminescence (Stevens, Dolan, & Greenberg, 1994; Kaplan & Greenberg, 1985). The transcription process is only triggered when the local density of AHL reaches a predetermined threshold, which serves as a go signal for the bacteria (Figure 5). There are several features used by the system to stop, that is, to prevent premature luminescence (Waters & Bassler, 2005). One feature is regulation of the stability of the constituent proteins: They are more stable when AHL is more concentrated (Zhu & Winans, 1999). Another is active pumping of AHL out of the cell: This process reduces cytoplasmic levels of AHL and thus dampens sensitivity until AHL concentration is high enough to overwhelm the pumping mechanism (Pearson, Van Delden, & Iglewski, 1999).
Several features of this stop/go process are notable here for the curious neuroscientist. First, the system implements a clock-like function by taking advantage of the consistency in reproduction rates of its own members. No member or subgroup serves as a clock or other timer function. In other words, the timing function is an emergent property of the system. Second, there is no centralized site that tells the bacteria when to glow; each individual agent makes up its own tiny mind, but because they are in the same environment, their activity is effectively coordinated through the localized cross-signaling of individual cells. Third, the system implements a specific and precise threshold-crossing process (a simple rule based on concentration levels of AHL), even though no abstract decision variable is calculated or represented. Finally, there is no need for any kind of modular self-control or inhibition. The lack of glowing is simply a consequence of the fact that there are insufficient concentrations of chemicals to drive the glowing; inhibition in this system is an emergent process (cf. Hampshire & Sharp, 2015).
Intriguingly, self-control in humans could arise out of a quorum sensing process. As with ants and bees (see below), the quorum sensing process can produce clock-like behavior. If self-control is conceptualized as a type of race to threshold model, this could be implemented by competing quorum sensing process with different accumulation rates. Ants and bees provide a direct model of such systems and are able to make flexible speed accuracy tradeoffs as well as risk avoidance for nest site qualities. In either case, it is the interaction of scouts sharing information on possible nest/hive locations as that produces both the conflict signal and the inhibitory control signal via the competing quorums.
Speed–Accuracy Tradeoffs: Ants
A decision made without taking the time to gather all the evidence may not be as accurate as a deliberate one, but it will have the virtue of speed. If time is costly (as when faced by an attacking predator), it may be worth going for the first good response, but if the decision-maker has all the time in the world, it's probably worth doing some pondering. Speed–accuracy tradeoffs are a staple of cognitive psychology (Bogacz, Wagenmakers, Forstmann, & Nieuwenhuis, 2010). Like humans and animals, many distributed decision-making systems make speed–accuracy tradeoffs, including slime molds (Physarum polycephalum) and honeybees (Apis mellifera; Dussutour, Latty, Beekman, & Simpson, 2010; Passino et al., 2007).
When looking for a new nest, individual ants (Leptothorax albipennis) leave the nest and evaluate potential locations within a few square meters (Franks, Dornhaus, Fitzsimmons, & Stevens, 2003; Franks, Pratt, Mallon, Britton, & Sumpter, 2002). These ants prefer to live in small colonies in thin cracks in rocks and are therefore easy to study in laboratory conditions. An ant that finds a potential nest site will recruit other ants to evaluate it by leading a tandem run back to the site. Thus, each site is evaluated by a large number of individuals, each of whom presumably makes a worse (less accurate) decision than the cumulative choice of several ants. Unlike bees (see below), individual ants appear to evaluate and compare multiple sites, giving them more individual knowledge and requiring smaller quorum sizes (Franks et al., 2002, 2003; Pratt, Mallon, Sumpter, & Franks, 2002). If enough ants appear at a single site, scouts recognize a quorum, and the quorum catalyzes a change in their behavior; scouts now carry their nestmates to the new site and deposit them there (Franks et al., 2002; Pratt et al., 2002).
This whole search and quorum sensing process is slow but accurate. But if the situation calls for a fast decision (such as during windy weather or threat of predation), the ant colony can make a speed–accuracy tradeoff (Franks et al., 2003). Each ant can reduce the threshold it uses to decide whether to switch from tandem run recruitment mode to carrying mode. The tandem run, being slower, allows other ants more time to discover other sites; the carry terminates the process more quickly. The ant itself does not know explicitly about the speed–accuracy tradeoff; it just has an internal sense of weather and adjusts its quorum sensing procedure—and the group's speed–accuracy tradeoff is an emergent consequence (Franks et al., 2003).
The neuroscience of the speed–accuracy tradeoff is not fully understood, but the parallels are easy to discern. It is believed that there is a threshold integration process for perceptual decisions (Bogacz et al., 2010). Recent work suggests it may involve changes in the baseline activity of neurons that serve as cortical integrators that bring them closer to threshold (Ivanoff, Branning, & Marois, 2008; Van Veen, Krug, & Carter, 2008), perhaps through disinhibition. Complementary research suggests that slower decisions involve inhibition from the subthalamic nucleus (Frank, Scheres, & Sherman, 2007; Aron & Poldrack, 2006). In either case, neurons encode a decision variable that, in a distributed manner, represents the evidence in favor of the decision. Although these models are not strictly distributed control models (because the thresholding is assumed to be separate from the accumulation), they have characteristics of it. A major goal of the stopping literature is to identify the key brain site that regulates stopping. The distributed control approach cautions that such a site may need not exist; instead of a site, there might be a neural mechanism at work, one that is not distinct from the sites of neurons that form the perception-action stream.
Conflict Detection and Resolution: Honeybees
Humans performing a cognitively demanding task may realize the task is harder than expected and devote more effort to it (Shenhav et al., 2013; Botvinick et al., 1999). For example, in the Stroop task, a participant is asked to either read a word or name the color of ink used to display it. Reading the word is easy, but naming the ink color, especially when the word itself is a different color, requires more cognitive control. Another form of conflict is decisional conflict, which is caused by ambivalence between two equally desired options (Strait, Blanchard, & Hayden, 2014; Cai & Padoa-Schioppa, 2012; Hayden, Heilbronner et al. 2011). Modular models of conflict detection and resolution generally involve a discrete conflict detector and resolver, which are often located in the dACC (Shenhav et al., 2013; Botvinick et al., 1999, 2001). We hasten to note that such models, especially with regard to dACC, are contentious: The signal may not be conflict per se, but in either case, it may regulate control, which is our interest here (Kolling, Behrens, Wittmann, & Rushworth, 2016; Shenhav, Straccia, Botvinick, & Cohen, 2016; Ebitz & Platt, 2015).
In springtime, thriving honeybee beehives reproduce. Roughly a third of the hive's members remain at the hive site and the others leave to form a swarm that gathers in one location and, in a few days, chooses a new hive site from a radius of several kilometers (Seeley, 2010; Camazine, Visscher, Finley, & Vetter, 1999; Seeley & Buhrman, 1999). Like our ants above, scouts evaluate promising nearby sites and then return and signal their quality with special dances (Figure 6). Dances indicating higher-quality sites induce other bees to investigate the same site. When scouts detect a quorum of bees at a site (typically around 20), they then return and provide a different signal, one that initiates a selection of the hive site by the swarm (Seeley, 2010; Seeley & Buhrman, 1999).
If there is one obvious best site, the decision will proceed quickly. But if there are two or more sites of approximately equal quality, the decision will proceed more slowly as the bees take the time to choose the best one. The swarm therefore is sensitive to decisional conflict: It monitors its own level of decisional ambivalence as the decision proceeds. Note that this conflict signal is a swarm, not individual variable; after all, no bee knows about more than one site, so no bee is conflicted. By not halting the search process, the swarm effectively recruits more processing resources (i.e., more bee search time) when conflict is high. As in mental effort, deliberation is not free; swarms are vulnerable to weather and predators, so there is an opportunity cost to delaying the construction of the hive (Lindauer, 1957).
Notably, the detection and resolution of conflict are emergent phenomena. No single bee that is sensitive to the conflict level—we know the rules the bees follow, and none of them deal with conflict. Nor is there a conflict signal represented in the bee's waggle dance or at any other point in the system. No bee has a specialized role before the swarm starts swarming. Still, the swarm as a whole is quite sensitive to decisional conflict and able to deal with it efficiently. It is also worth noting that an aggregate measure of hive activity, say, the number of active scouts or number of active dances during the decision, will show clear and strong aggregate conflict signals. This finding is intriguing because conflict signals are seldom observed in the activity of single neurons and yet are robustly observed in the brain's hemodynamic activity (see below).
The key point is that these signals are not read out. An important concept from a distributed view is that a high-level signal can emerge out of lower-level processes without ever actually being “read-out” or directly computed. In the case of the bees, the idea is that there is no observer reading out the conflict signal. It is a by-product of the process like the heat of a car engine. The enhanced processing that accompanies high conflict is a consequence of the rules of interaction of the bees, which do not involve any conflict monitoring at that level.
EVIDENCE FOR DISTRIBUTED EXECUTIVE CONTROL IN THE BRAIN
We turn now to the neuroscience of control. As noted above, there is a broad consensus that executive control is modular, not distributed (Botvinick & Cohen, 2014). We believe that one reason for relative unpopularity of distributed control systems by scholars is that they are unfamiliar and unintuitive. Indeed, distributed control is notoriously difficult for us to intuit. Terms like the “ghost in the machine,” “the invisible hand of the marketplace,” and “asking the hive mind” are reminders that our own minds naturally impute discrete and coherent agency even when dealing with mindless and ghostless distributed systems. Still, many distributed control systems are intuitive and can become more so with familiarity.
A Case Study: The dACC
To look at these general issues in detail, we will take the dACC (often just ACC) as a case study. The dACC is part of the cingulum, a band of cortex that wraps around the corpus callosum in the sagittal plane. The dACC receives a broad and diverse set of inputs that includes both limbic and cognitive regions, as well as dopamine signals, and projects to motor, premotor, and executive regions. These factors make it a natural site for serving as a monitor and controller. Indeed, a great deal of evidence links it to these two functions, among others. This evidence includes physiology (Heilbronner & Hayden, 2016), neuroimaging (Hayden & Heilbronner, 2014; Shenhav et al., 2013; Kolling, Behrens, Mars, & Rushworth, 2012), and lesion studies (Picton et al., 2007; Kennerley, Walton, Behrens, Buckley, & Rushworth, 2006; Rudebeck, Walton, Smyth, Bannerman, & Rushworth, 2006). Most prominently its responses are activated by contexts that promote control (Shenhav et al., 2013; Rushworth, Noonan, Boorman, Walton, & Behrens, 2011). These include conflict (Ebitz & Platt, 2015; Botvinick et al., 1999; but see Ebitz & Hayden, 2016; Kolling et al., 2016), surprising and unexpected outcomes (Hayden, Pearson, & Platt, 2011; Matsumoto, Matsumoto, Abe, & Tanaka, 2007; Ito, Stuphorn, Brown, & Schall, 2003), rewards (Hayden, Pearson, & Platt, 2009, 2011; Kennerley, Dahmubed, Lara, & Wallis, 2009), progression through a task (Ma, Hyman, Phillips, & Seamans, 2014), changes in environmental context and volatility (Behrens, Woolrich, Walton, & Rushworth, 2007; Procyk, Tanaka, & Joseph, 2000), control of actions (Strait et al., 2016; Nakamura, Roesch, & Olson, 2005), and others not listed here. It is also directly activated by factors related to control, such as persistence (Blanchard, Strait, & Hayden, 2015; Chudasama et al., 2013; Parvizi, Rangarajan, Shier, Desai, & Greicius, 2013).
These findings generally support a modular view of cognition, one in which dACC takes on the specialist role of monitor and controller. However, a broader review suggests that dACC is neither uniquely involved in monitoring and control nor is its function primarily these two roles. Indeed, the very long list of functions above should raise suspicion for a devotee of the modular viewpoint. Yes, these variables can all be placed under the rubric of monitoring and control, but at some point the definition becomes so elastic that it contains almost all of cognition. Second, are all these functions found only in the dACC? Unlikely. Many regions (e.g., striatum, dlPFC, insula) are coactive with dACC in control tasks forming functional networks (Dosenbach et al., 2007). Most of these functions are shared with many other brain regions (Cisek & Kalaska, 2010). For example, recent work points to the importance of the orbital surface in classically anterior cingulate functions like conflict monitoring and resolution (Mansouri, Buckley, & Tanaka, 2014), and regulating the explore–exploit tradeoff (Blanchard, Hayden, & Bromberg-Martin, 2015).
Studies that compare dACC activity with other brain regions often find that differences are more qualitative than quantitative (Azab & Hayden, 2016; Hosokawa, Kennerley, Sloan, & Wallis, 2013; Kennerley et al., 2009). That is, single-unit recordings often find similar task variables encoded in activity of ACC and other prefrontal regions. The specific proportions of cells coding these variables may be different, but their presence or absence is, in many cases, not a distinguishing factor between regions. Indeed, control is associated with many other prefrontal structures, including OFC, dlPFC, vmPFC, and vlPFC (e.g., Wilson, Takahashi, Schoenbaum, & Niv, 2014; Buckley et al., 2009; Schoenbaum, Roesch, Stalnaker, & Takahasi, 2009; Bechara, 2005). Nor are these functions limited to pFC; control signals are observed in the parietal cortex, the posterior cingulate cortex, the thalamus, and the striatum (e.g., Hayden, Smith, & Platt, 2010).
More broadly, summaries of dACC function tend to emphasize its potentially specialized role as a hub, linking visceral, cognitive, and motor systems (Heilbronner & Hayden, 2016; Rushworth et al., 2011; Bush, Luu, & Posner, 2000; Morecraft & Van Hoesen, 1997). But is it really all that specialized? There is anatomical and functional evidence for its hub nature, but it is also true of other brain regions, including, for example, posterior cingulate cortex (Heilbronner & Platt, 2013; Heilbronner, Hayden, & Platt, 2011) and insula. Indeed, rich interconnectivity is a feature of many brain systems (Heilbronner, Rodriguez-Romaguera, Quirk, Groenewegen, & Haber, 2016; Wang & Kennedy, 2016; Heilbronner & Haber, 2014).
Nor are the response properties observed in the dACC uniquely control-related. Many of them seem to fit naturally into the category of stimulus–response processing, rather than as a regulator of that processing. That is, if we think of the brain as a system that converts sensory inputs to motor outputs, we should expect in a modular brain to find no sensory and motor signals in dACC and instead find pure control-selective signals (Cisek, 2012). Instead, dACC is prominently responsive to both sensory stimuli and to actions. Encoding of sensory feedback often corresponds with activation of cortical body maps related to feedback modality, indicating an embodied processing (Procyk et al., 2014). One convenient parameter to look at is spatial representation; this is a prominent property of the physical world but should, in theory, not be part of the recondite world of control. And yet dACC encodes the locations of stimuli under consideration and the specific details of actions (Strait et al., 2016; Hayden & Platt, 2010; Luk & Wallis, 2009; Isomura, Ito, Akazawa, Nambu, & Takada, 2003).
Consider, for example, the OFC, a region that is often thought to have a basic value-processing role and is not part of standard control network models. Yet, recent evidence supports the idea that some control signals, including working memory, switching, selection, and rule maintenance may occur locally (Sleezer, Cstagno, & Hayden, in press; Sleezer & Hayden, 2016; Chase, Tait, & Brown, 2012; Lara, Kennerley, & Wallis, 2009).
Together, these pieces of evidence argue that the differences between the dACC and adjacent structures are not as strong as is conventionally believed. Evidence from fMRI data has begun to emphasize functional network relations over localized specialization during task performance (Sporns & Betzel, 2016). As a relevant example, many factors that activate dACC also activate pSMA (e.g., Jahn, Nee, Alexander, & Brown, 2016). This suggests instead there may be continuity of function between dACC and its neighbors and afferents. The broad functions, especially in the control domain, that it serves are more distributed than modular. Moreover, the units of dACC—its neurons—appear to play a role in input–output processing as well as in generation of control signals.
If the analogy from individual baboons to neurons were to hold, we might then expect that single neurons would not encode the conflict associated with, for example, difficult choices. Rather, neurons would code the parameters of specific choices individually, and the conflict signal would arise emergently from the interactions of many neurons. Just such a theory is proposed by Nakamura et al. (2005). In that study, participants performed a stopping task and, on difficult trials, showed behavior signs of conflict adaptation. However, no neuron encoded conflict explicitly. Rather, the scientists were able to show that the properties of the population would lead to conflict correlates in the aggregate.
Maybe Executive Control Could Be Distributed in the Brain?
A priori, it is not unreasonable to think so. A basic description of the brain sounds like an ideal candidate for a distributed control system. Neurons are agents that can only communicate with a very small number of neighbors relative to the whole population. Like bacteria, they use a variety of diffusible chemicals to communicate. Each neuron can monitor an extremely limited portion of the world and can broadcast its signals to a very narrow part of the world as well. Each neuron has limited but powerful and nonlinear computational properties.
Moreover, each cell is autonomous, but they work together, noncompetitively, in the service of a much larger goal. Individual neurons possess the ability to regulate the activity of other neurons (or output structures) through changes in firing rate. This activity can serve as both a processing and a regulatory role. The properties of the whole system (the brain) are rich and flexible, much more so than any of its constituents (Hofstadter, 1985). The brain makes use of both positive and negative feedback and shows slow changes over time.
Strong circumstantial evidence for the distributed view comes from lesion studies (Wilson, Gaffan, Browning, & Baxter, 2010). Damage has surprisingly weak and graded effects; graceful degradation is a well-known property of distributed systems (McClelland, Rumelhart, & PDP Research Group, 1987). Of the major “clean” effects associated with lesions (prosopagnosia, hemianopia, scotoma, and so on), few would be considered executive control effects. Instead, impairments in executive control can come from lesions in many different areas, and associated effects are generally graded and only grow serious when the lesions become quite large (Wilson et al., 2010; Farah, 2004).
Although there is some evidence for control-specific lesions (Levine et al., 1998; Duncan, Emslie, Williams, Johnson, & Freer, 1996; Shallice, 1982), it may be difficult to pin these data clearly to control functions. In this case, we are defining control to mean a signal that determines the amount of resources necessary for a given task. Instead, it may be that more difficult processing requiring greater neural resources is impaired, whereas simpler processing requiring fewer resources is spared. Consider, for example, an ant colony with a large proportion of members lesioned. That colony would have no trouble choosing a hive site if the decision was easy but would have a great deal of trouble with a more difficult decision. We should not then conclude that the task difficulty module is broken.
Indeed, the brain was the original inspiration for connectionist and PDP networks. The linkage between brain organization and other distributed control systems has been pointed out by many others before (Seeley, 2010; Couzin, 2009; Mitchell, 2009; Passino et al., 2007). Given these facts, it is striking that the distributed view has not continued to serve as the null hypothesis for modular theories as a viable alternative view.
Advantages to a Distributed Control System
From the perspective of adaptiveness, there are several advantages of a distributed control system with simple agents (Brooks & Flynn, 1989). First, because it is self-organized, there is no need to build a special centralized organization system that will link up control elements with their corresponding processors. A modular system requires the equivalent of a telephone switchboard; a distributed one does not. Second, that self-organization gets around the specter of infinite regress (Cooper, 2010). For example, if we have a special centralized organization system, we need another system to build and maintain it, to monitor its functioning, and so on, ad infinitum. Self-organizing systems are easier developmentally—there is no need to prespecify their organization genetically or any other way. They are also more robust to damage and can more readily adapt and be amenable to plasticity, as such occurs with learning. They are generally more flexible for novel situations. Finally and most important, distributed control is a good way to get complex and adaptive behavior from systems consisting of elements that are less complex (Sumpter, 2006). From a theoretical perspective, distributed system makes sense. Many brain functions are distributed, including perception and object recognition, storage of episodic memories, motor planning and execution, and, arguably, economic decision-making (Strait, Sleezer, & Hayden, 2015; Cisek, 2012; Cisek & Kalaska, 2010).
We do not mean to imply that no current work could be classified as distributed. Quite the opposite is true. Many models have distributed aspects (e.g., Behrmann & Plaut, 2013; McClelland et al., 2010; Botvinick & Plaut, 2004, 2006; Botvinick et al., 2001). And it is worth reiterating that the modular and distributed views are not mutually incompatible. In reality, they may exist on a spectrum. Executive control may be heterogeneous; some aspects may be modular, whereas others may be distributed and these differences may occur within a particular region. Instead, our major goals are to highlight the key distinguishing features of distributed and modular systems to provoke a reconsideration of preconceptions.
In many studies (including, we hasten to admit, many of our own), we pick out some psychological process of interest. We then ask whether brain activity in some neuron or voxel within a given brain region correlates with a measure of that variable. If we get a positive result, the simplest step is to infer that that variable is reified in the brain. The distributed perspective cautions against this strategy; such correlations may be real but may only correlate with emergent properties of the system. And if the underlying processes are dissimilar, we will draw false conclusions. In other words, we are always in danger of reifying higher-level processes at the lower level.
Avoiding this pitfall will require careful consideration of both distributed and modular implementations of circuit dynamics and network activity, as well as attempts to interpret data that conflict with preconceptions. Brains are complex distributed systems, and they face many of the same constraints as others. It should not be surprising that they have a great deal in common with ant colonies, bee swarms, and herds of migrating baboons (Seeley, 2010; Couzin, 2009; Passino et al., 2007; Sumpter, 2006; Hofstadter, 1985, Ch. 25, 26).
This work is supported by a CAREER award from NSF (BCS1253576) and an R01 from NIH (DA038615) to B. Y. H. R. A. is supported by Research Fellowships for Young Scientists from the Japan Society for the Promotion of Science. We thank Tom Seeley for patient explanations and Amanda Oglesby-Sherrouse for introducing us to the wonderful world of bioluminescent bacteria.
Reprint requests should be sent to Benjamin R. Eisenreich, Brain and Cognitive Sciences, Meliora Hall, University of Rochester, Rochester, NY 14627, or via e-mail: firstname.lastname@example.org.