Abstract

Life and cognition are inherently circular dynamical processes, and people have difficulty understanding circular causation. I give case studies illustrating some resulting confusions, and propose that the problems may lie in failing to properly distinguish between similar concepts used to describe both local and global features of a system. I analyze how explanations in terms of circular causation work and how they rely on principles of normal settlement. Even though they typically will not explain the origins of phenomena (that is the province of linear causal explanation), circular explanations have predictive power for any persisting (i.e., stable or metastable) phenomena.

1 Where to Start?

Artificial life owes much to cybernetics. The influential 1940s–1950s Macy conferences referred to cybernetics as “circular causal and feedback mechanisms in biological and social systems.” McCulloch, the conference chair, described his quest as “what is a number, that a man may know it, and a man, that he may know a number?” [29]. Change “number” to “thing” and “man” to “organism” for the circular core of autopoiesis [37]: How can an organism and its world co-define each other? Circular causation is central to understanding cognition, whether biological or artificial.

Linear causation and explanation is familiar. One starts with a firm foundation of agreed facts, and systematically builds up from there. However, circular causation is like a Sudoku puzzle where no part of the whole is guaranteed until all the interlocking constraints can be simultaneously satisfied. It is not obvious where and how to start.

Here I show examples of typical traps people fall into when they attempt to understand circular causation. Though I aim ultimately at the circularity of full-blown cognition, I start with a minimal non-cognitive example of circular causation, DDWFTTW (direct downwind faster than the wind). Despite its minimality, many find this challenging to understand. I use this, together with an even simpler example of a natural rock arch, to illustrate the basic concepts of circular causation.

Then I progress through a number of further examples extending into the cognitive domain. These are used to illustrate how explanations based on circular causation work, what principles they are based on, what limitations they have. I identify some patterns in the common errors made.

It is astonishing how little is taught about circular causation despite its centrality to so many important fields (especially biology, cognitive science, and artificial life) and despite the abundance of confusion and fallacies arising from widespread unfamiliarity with the concept. Areas of study with the strongest track records for discussing circular causation include cybernetics [40] as well as autopoiesis and enaction [3739]; but even these do not focus on the particular confusions and misunderstandings I address here. Hence this article presents analytic studies with a didactic motivation; in my opinion there is a need for such education. It is extended from an ECAL conference paper [20], and the text and illustrations borrow freely from there.

1.1 DDWFTTW: Direct Downwind Faster Than the Wind?

Consider the machine of Figure 1, with but a single component moving part. A boat is constrained to run left or right along a canal. The single shaft shown is positioned in a sleeve and rotates freely according to the forces transmitted through the air and water propellers. From the densities of air and water, the propeller details, and the drag resistance of the boat, in principle we can calculate what steady state boat velocity results for a given wind velocity. Our minimal agent has but a single class of behavior, steady motion, powered by the relative movement between wind and water.

Figure 1.

Can it go downwind faster than the wind? Yes.

Figure 1.

Can it go downwind faster than the wind? Yes.

Many readers may doubt that for any parameter settings the boat will move downwind faster than the wind. YouTube videos (usually showing land-based but equivalent versions: search for “DDWFTTW”) have many comments claiming to prove the impossibility. We are told that the director of the Lawrence Berkeley National Laboratory refused to allow an external speaker to present a talk on this on the grounds that such a vehicle would violate the laws of physics.1 The reasoning usually starts in the most obvious starting place by considering the wind driving the air propeller (based on the velocity relative to the boat) and thereby driving the boat. If it accelerates up to 10 km/h, the relative wind velocity drops to zero, surely leaving no further force available to propel the boat to a higher speed?

This reasoning is wrong, despite the mechanism being so simple and the intuitions so compelling. The actual steady state solution has the shaft rotating in the opposite direction to that normally assumed. That is, if the handedness of the air propeller is such that wind from the left would drive an unconstrained version clockwise, in fact its coupling with the water propeller results in it rotating anticlockwise. A linear chain of reasoning starting from the water propeller likewise does not immediately generate this solution, since with an initially stationary boat there is nothing to drive that propeller. The linear reasoning does not fail from starting in the wrong place—there is no right place to start! Circular explanations only work in translation to linear reasoning when they include the full circuit of component processes jointly maintaining each other in steady state.

For a full analysis one has to calculate the torque on each propeller (at steady state these torques sum to zero) and the linear forces on each propeller (which at steady state sum to a total that equals the drag on the main hull). Formally, we have boat speed B and shaft rotation R such that
$dB/dtisanincreasingfunctionofThrustwater+Thrustair+Drag$
(1)
$dR/dtisanincreasingfunctionofTorquewater+Torqueair$
(2)
where all the terms are themselves functions of B, R, and propeller properties such as thrust coefficients, torque coefficients, advance ratios, efficiency, size, and handedness. The details are too complex to explore here. But it can be shown [16] that a whole range of parameter choices allow this apparently paradoxical boat motion downwind faster than the wind—just as different parameter choices will allow boat motion upwind.

Navigators of sailing boats will be more familiar with these issues than most. As well as being familiar with the ability to tack into the wind (progressing through zigzags at around 45º to the oncoming wind), they will know that the fastest way to sail downwind is to tack on a broad reach at around 45º to the downwind direction. This tactic is used in modern high-speed racing, particularly with ice yachts [8]. Despite the extra distance involved in such tacking, the speed made good in the downwind direction can outpace the wind by a significant factor. If one transplanted such a speeding ice yacht, on a downwind reach (or upwind tack) on a flat expanse of ice, to some imagined large ice tube with the same wind blowing down (or up) it—we conveniently here ignore gravity—such an ice yacht would spiral down the tube faster than the wind itself (or up the tube against the wind). The sails of the yacht would act much as the blade of a propeller, and the circular motion of this blade translates into linear motion downwind (or upwind). Combine such possibilities with the two propellers of Figure 1, and with the energy source from differential motion of air and water; then downwind faster than the wind becomes easily achievable [16].

Why do so many people find this paradoxical and hard to believe? Firstly, because of unfamiliarity with the circular causal reasoning required, in terms of the attractors of steady-state dynamics. But a second, related reason may be the easy ambiguity of using terms such as “drive” both locally (wind drives propeller) and globally (wind drives boat) but failing to realize these are different senses. This is a version of the mereological fallacy [7], or more generally a basic category error. Pre-Copernican astronomy was likewise misled by confusing motion locally relative to Earth with a supposed global motion relative to some universal framework.

1.2 Linear Causation; Chickens and Eggs

We assume here that the motivation for wanting an explanation is to be able to predict events, or in other words to avoid surprise. The default form for an explanation for a phenomenon is in terms of linear causation: “We accept without question premises A and B, and we can then deduce consequence or effect C; subsequently, from A, B, and C we can deduce D, and thence E, F… in linear sequence.” This is the way we build a tower out of bricks in successive stages on a firm foundation, or a mathematical proof derived from axioms. For each phenomenon, for each brick in the tower, for each lemma in the proof, we can provide lines of reasoning or of support back to firm foundations. An annoying young child, repeatedly asking “Why?” to each proffered new level of explanation, is appealing to this form of causal explanation. They may be seeking the foundational assumptions, which ultimately can only be given as “Because!”; or they may just be being annoying.

Such forms of explanation are so common that we usually omit the “linear” qualification and assume that all causal explanations will take this form. Hence the perceived need for an answer to “Which came first, the chicken or the egg?”. Explanations of origins must necessarily be linear through time, must appeal to linear causation. But in contrast to this, we here start to move towards systems of circular causation, where a number of components are mutually interdependent. This puzzles some, as we shall see.

1.3 Irreducible Complexity and Rock Arches

Creationist advocates of “intelligent design” focus on what they call the “irreducible complexity” of biology, where there is their version of circular causation: “…a single system which is composed of several well-matched, interacting parts that contribute to the basic function, and where the removal of any one of the parts causes the system to effectively cease functioning” [6, p. 39]. Their issue is not so much with the functioning or persistence of such systems as with their origin. If the parts are mutually interdependent, how could they have been assembled incrementally through successive viable stages, such as implied by a Darwinian explanation?

The answer is illustrated by natural rock arches (Figure 2), which, though simple, nevertheless fit the definition of irreducible complexity.2 There are mutually dependent component parts (e.g., for a simple choice of segmentation, the left and right halves L and R) that do indeed mutually support each other; remove either one and the other shifts in consequence, probably with the collapse of the arch. There is a further component, the underlying ground G, that is also necessary to provide support; but G is independent of L and R, since the removal of either leaves G unchanged. The natural processes that formed the arch did not require the hand of a creator during assembly; successive stages of addition (deposits of sediment) followed by subtraction of “scaffolding” (erosion) are sufficient. Darwinian evolution is similarly not constrained by the paucity of imagination of the creationists, and is quite capable of assembling through successive viable stages of subtraction, neutral change, and exaptation as well as addition. The fact that the maintenance of an end product may be explained through circular causation is no barrier to a separate linear causal explanation for its origin.

Figure 2.

Irreducible simplicity? Natural rock arches: the world's widest (Landscape Arch, Utah, 88 m wide: Thomas Wolf, www.foto-tw.de, Wikimedia Commons, CC BY-SA 3.0) and the world's tallest (Shipton's Arch, Xinjiang, 460 m tall, IH).

Figure 2.

Irreducible simplicity? Natural rock arches: the world's widest (Landscape Arch, Utah, 88 m wide: Thomas Wolf, www.foto-tw.de, Wikimedia Commons, CC BY-SA 3.0) and the world's tallest (Shipton's Arch, Xinjiang, 460 m tall, IH).

2 Circular Causation as Used in Explanations

The arch provides an intuitive picture of circular causation; let's formalize this. In particular, we characterize when and how explanations of a phenomenon can be framed in terms of circular causation, and what are the limitations.

We use explanations to reframe complex phenomena or patterns, possibly difficult to understand, in terms of simpler entities or components that we do indeed comprehend. We can build an explanation on the known effect that each component A, B, C, …, Z has on one or more of the others. If such causal chains feed back on themselves in circuits A → B → … → A, then we have the potential for circular explanations. If such circuits can be broken down into independent subcircuits (e.g., A ⇿ B and C ⇿ D), then those can be analyzed independently. Hence it is sufficient to just consider sets (A, …, Z) where each member is directly or indirectly potentially affected by every other member.

The rock arch is a minimal example, where each pier (L and R) can take on one of two values: say 1 for standing and 0 for collapsed. There are two possible steady stable states for {L, R}: {1, 1} for a standing arch, and {0, 0} for complete collapse. We are assuming that the lean of each pier is such that neither can stand unsupported, so, though we can imagine an instantaneous snapshot taken of situation {0, 1} or {1, 0}, neither is a viable long-term configuration.

Generalizing from this, we can say that a circular explanation involves explaining the existence of (one or more) attractor(s) in a system of several interacting components with circular chains of cause and effect; all components, both individually and jointly, need to be in stable equilibrium. Such an equilibrium may be a static point attractor, as with the arch, or a steady state of dynamic equilibrium, as with DDWFTTW, or indeed any other form of cyclic or even chaotic attractor. Such an explanation is useful even where we may not have a linear historical explanation of origins available; we can still predict how external perturbations will shift or collapse such a system. Such explanations inevitably have no obvious starting point, and we shall see below how this can be confusing.

2.1 Time Scales and Circular Causation

Any physical system that displays patterns significant enough for us observers to contemplate and discuss must inevitably have some associated rough range of time scales. Circular causation, being at root based on the concept of attractors, is no different. For natural rock arches the range might lie roughly from a few seconds to many years. An unsupported pier might take a few seconds to collapse, so one needs a longer period of observation than that to be happy that an arch is stable. At the other end of the scale, the fact that all such arches will have changed or disappeared after millennia does not stop us calling them currently stable over many years.

Typically we ignore events faster than the fast end of this time scale, except in so far as they will destabilize any unstable equilibrium. We may call this noise; it ensures the dynamics heads into attractor basins. But at the top of the range we may often choose a variety of time scales for consideration, when we consider metastability and the difference between variables and parameters. On a time scale of years, rock erosion can probably be ignored and we can treat a rock pillar or arch as stable and fixed. When however we shift our perspective to that of millennia, erosion becomes a variable that we must consider significant. A parameter typically describes an effect that counts as a variable on a long time scale, but can be considered as fixed over a shorter time scale. It is often necessary, when considering explanations in terms of circular causation, to make explicit such shifts in time-scale perspective. This may become clearer in the Daisyworld example below, where insolation is considered a fixed parameter for the purposes of temperatures settling into an attractor—but nevertheless may vary over longer time scales.

2.2 Sequential and Steady State Causation

In linear causation, where A has an effect on B that has a knock-on effect on C, …, effects follow causes sequentially in time. At first sight this sequential property seems incompatible with circular causation.

But not really, since circular causation is based on steady state analysis, and sequential cause-and-effect is crucial to understanding whether a circuit A → B → … → A in steady state is stable or unstable. The reasoning involves considering a random small shift to one component, say A, when the whole circuit is in some specific steady state situation, and then following the consequential chain of cause and effect around the circuit until it reaches A again. If a small positive (or, alternatively, negative) shift at A results in a consequent negative (or, respectively, positive) response on A, the circuit is in negative feedback and hence stable. With the reverse response, it is unstable. So the analysis in terms of stable and unstable steady states that circular causation usually relies on is at a higher conceptual level than linear cause-and-effect alone, but nevertheless builds upon such sequential linear chains; there is no incompatibility.

It might be suggested that A → B → A should be rewritten as something like A(t) → B(t + 1) → A(t + 2), for the update ticks of some universal time t. I should emphasize that this would be a highly undesirable misinterpretation. It confuses the map with the territory—confuses a computational simulation of a world (that typically uses clock ticks) with the real world (that does not). The reference in the previous paragraph to the consequences of “a small shift” in one variable should be read as “an arbitrarily small shift, over an arbitrarily small time.” Mathematically speaking, we learn in calculus to define dx/dt as the limit, as δt → 0, of δx/δt, where x takes value x + δx at time t + δt. Stability of x at time t is defined as dx/dt = 0 at time t (and not at any time t + 1).

Why does this apparently insignificant distinction raise any concerns? Because the failure to appreciate it has led to too many simulations, in artificial life and elsewhere, failing to correctly model their target worlds through inappropriate choice of update time steps. This typically results in computational artefacts in the simulation, such as numerical instabilities, that mislead; we shall see examples in the Daisyworld (Section 4.2 below). One pragmatic way to ensure that the simulation choice of update time step δt is small enough is to make a first guess of an appropriate δt and run the relevant part of the simulation, recording the result; then repeat with a δt that is 1/10 the size, then again 1/10 smaller, …, until the results do not materially change in value. This corresponds closely to the literal interpretation of defining dx/dt as the limit, as δt → 0, of δx/δt. Simulations are invalid if there has been no check as to whether an inappropriate choice of δt has been made; at minimum a basic reality check is needed; ideally this more principled check3 will use decreasing values for δt.

3 Circularity Taken to Extremes: Holograms

The simplest forms of circular causality involve a small number of mutually dependent variables, as in the mutually supporting piers of a stone arch, or the dynamical relations of a boat with propellers. More complex circularity can arise with multiple interconnected units forming multiple overlapping circuits. Thomas and colleagues [3335] present interesting perspectives on these, particularly in the biological context of cell differentiation. What happens when you take this towards the extremes of millions of tiny interacting units, approaching an analogue continuum?

One version of such an extreme is illustrated by a hologram [15], Figure 3. The viewers looking through the holographic film see an image behind, in 3D, that offers them the same shifts in perspective as they move around as the shifts when they are viewing the original. Whereas in normal photography different parts of film record different parts of the image, a crucial feature of holograms is that the whole of the image is recorded everywhere across the film. So if, for example, only a small part of the film (shown at lower left in Figure 3) is available, then the whole of the image can be viewed (at somewhat lower resolution) by peering through that segment at an appropriate angle.

Figure 3.

The virtual statue appears to the viewers to be in 3D behind the holographic film, and image parts, such as the torch, have the same relationship to the whole as in the real statue. but there is no direct one-to-one correspondence between any small segment of the holographic film and some segment of the image.

Figure 3.

The virtual statue appears to the viewers to be in 3D behind the holographic film, and image parts, such as the torch, have the same relationship to the whole as in the real statue. but there is no direct one-to-one correspondence between any small segment of the holographic film and some segment of the image.

This property means that the holographic film has the useful feature of graceful degradation, that loss of parts of the film does not lose any part of the image. The whole is still available, though the image quality starts to degrade. Thus, the hologram model has been very influential as a source of inspiration for how brains may facilitate the storage of memories [14, 22]. The metaphor suggests that—just as, with holograms, there is no one-to-one mapping between image features and locations on the celluloid film—there is no need for any one-to-one mapping between individual memories and any locations in the brain that might be “storing” those memories. These ideas lie at the root of recent deep learning advances in AI; Hinton, recipient of the 2018 Turing Award for his foundational contributions to deep learning, was influenced in his original research directions by exactly this hologram metaphor [14].

In what sense do holograms involve circular causation? Appropriate laser light focused on the interference patterns recorded on the celluloid film results in the creation of a light field, or optic array, that is in crucial aspects identical to the light coming from the original object. It is instructive (and somewhat counterintuitive) to identify what is happening by reference to Figure 3, and the role of the observers depicted there. These observers will agree on the 3D location of the virtual image; if they each point at it, those pointer lines will intersect at that 3D location. Yet if we go round behind the film, nothing is there; we pass our hand through empty space. There are not even any relevant light waves emanating from that specific location. In the absence of the observers there is nothing there.

However, in the presence of those observers, the reconstructed wavefronts [15] resulting from the constructive and destructive summation of light waves from everywhere on the available image film will mutually support each other to form stable perceptions by those observers, perceptions that are interpreted by them as if there were an object there. The agreement by the observers as to the location of the virtual image remains stable even as the observers shift their viewpoints, even as parts of the celluloid film are removed or corrupted.

The stability, in location and form, of the virtual image (despite noise on or partial removal of the celluloid film, despite observer movement) follows the same pattern as the stability in location and form of the stone arch (despite any ground tremors). But whereas the stone arch is composed of real stone at its actual location, the virtual holographic image is not composed of anything material at its location, not even from light waves arising from there. The virtual image arises from the collaboration between (a) the observers with their visual perception attuned to normal visual circumstances and (b) the optic array from the celluloid film illuminated by the laser. Just as left and right pillars of a stone arch support each other, here (a) observers and (b) that optic array support each other to maintain a stable virtual image at a location where nothing really exists.

Stability of something as sophisticated as the perceived form and location of a virtual image in the eyes of a human observer might seem far distant from the stability of an arch. But they are both examples of stable equilibria within circular causation A → B → A. Look for the consequences of noise, look for invariants despite such noise. In the above sketch of the hologram the only noises considered so far have been disruption of the celluloid film and observer movement, as illustration of the principle; a fuller analysis would include further consideration of the human perceptual system.

4 Moving towards Agent-Based Circular Causation

Our first two examples, DDWFTTW and the arch, were inanimate, as is the hologram. We now move towards more animate models of circularity. Living creatures both depend on the world around them, and have effects on that world, and hence entail circular feedback loops.

We start with basic environmental interactions, in a simple yet still counterintuitive artificial life model. From there our examples progress to more full-blown examples of cognition.

4.1 Daisyworld: How Can Opposite Effects Both Be Good for You?

Our boat example of circular causation has just one attractor to its dynamics, for a given set of parameters. Our next case study, Daisyworld [18, 19, 41], has several potential attractors and introduces (at a simplistic level) notions of viability and homeostasis.

As summarized in Figure 4, the model assumes that daisies on a gray planet have a limited range of viability dependent on their local temperature. A gray daisy derives its temperature from the sun as it alters in solar output (over centuries), and it is viable over a limited range of such external perturbations. A black daisy, by absorbing extra heat, extends its range right (to less sun); a white daisy, reflecting heat, will extend its potential range left (to more sun); the extension of range in one direction is not penalized by a corresponding diminution in the other direction.

Figure 4.

Black, white, and gray daisies have the same (green) viability dependence on local temperature. External forcing from the Sun varies. Black increases and white decreases the local temperature. Feasibility ranges (⇔) of both black and white are extended to lower/higher perturbations.

Figure 4.

Black, white, and gray daisies have the same (green) viability dependence on local temperature. External forcing from the Sun varies. Black increases and white decreases the local temperature. Feasibility ranges (⇔) of both black and white are extended to lower/higher perturbations.

The core result of Daisyworld theory, as I have argued elsewhere [18, 19], can be summarized with a single-daisy version thus:
$dDdt=HT−D.$
(3)
For a current local temperature T, the daisy population D moves towards some value specified by a non-negative viability function H(T). Let the viability range S be the support of H—in mathematical terms the range of T-values for which H(T) is nonzero—the range of temperatures for which daisies are viable. We have
$dTdt=P+kD−T.$
(4)

For some externally fixed perturbation P (e.g., derived from the current solar output) and the current daisy population D, the temperature moves towards a value influenced by the quantity of daisies D—upwards if k > 0 (black daisies absorb heat), or downwards if k < 0 (white daisies reflect heat). This coupled pair of equations in T and D will have one or more stable equilibrium solutions with D either zero (extinct) or positive (extant). For any given values of k and P we define Qk(P) as the maximum D value at any such stable equilibrium. The feasibility range of k, F(k), is the support of Qk(P), the range of P-values for which there is at least one stable equilibrium solution with positive D. We can show [18, 19] that when k = 0, F(0) = S is the support of H; though F(0) is a feasibility range of P-values and S a viability range of T-values, here they happen to be measured in the same units (though in more complex Daisyworlds they will not). But the important Daisyworld result is that for all nonzero k, F(k) ⊇ F(0) and there is always some finite, big enough k = K (whether positive or negative) such that F(K) ⊃ F(0). In simple terms, make k big enough (positive or negative), and you will increase the feasibility range; whatever k, it is impossible to reduce the feasibility range below that for k = 0.

For a simple demonstration we may use a simple hat-shaped viability function:
$HT=max01−2T.$
(5)
This is a “witch's hat” shape, with a peak value at T = 0 decreasing to zero for T anywhere outside the viability range [−0.5, 0.5]. I give here4 basic Matlab code for numerically simulating the above equations (3)(5). Though this is the simplest example with just one daisy type, it serves as an introduction to the core principles before advancing to more complex Daisyworlds with two daisy types or more.

In this sense of promoting increased feasibility range, both increasing (black) and decreasing (white) local temperature are “good” for the viability of an otherwise neutral daisy. The literature is full of people who disbelieve such a counterintuitive result; some of the most prominent are cited and discussed by Harvey [18, 19], pointing out that this is largely due to misunderstanding of the circular causation. In particular, there is a tendency to confuse the term viability, which refers to an individual daisy (shown in green on the vertical axis in Figure 4) with what is here called feasibility, referring to the potential viability within a range of external perturbations (shown in purple on the horizontal axis); these are (literally) orthogonal concepts.

So again, this may be in part due to a pre-Copernican confusion between local and global concepts.

4.2 Daisyworld and Numerical Instabilities

Simulations of Daisyworld can mislead if they are not constructed so as to pass the reality check discussed above in Section 2.2 on sequential and steady state causation. A classic example comes from Kirchner, a Gaian skeptic. In a review of Daisyworld scenarios [26], he described one (his Figure 3 and associated text) that exhibited “pathological” behavior, an unstable oscillation between only-black and only-white daisies. My simulation version of this scenario came to a stable (and non-pathological) equilibrium with intermediate values, as expected. In an attempt to understand how our simulations differed, from correspondence with Kirchner (personal comm., May 2015) I established to my satisfaction (if not to his) that this was a classic case of the simulation time-step failing a basic reality check. If
$dBdt=W−B,dWdt=B−W$
(6)
is simulated by Matlab code such as

with a suitably small value for the time-step delta, then [B, W] settles down from initial values [1, 0] towards equilibrium at [0.5, 0.5]. But if delta is set to a value 1, the behavior does indeed oscillate “pathologically” between [1, 0] and [0, 1]. The former, non-pathological version is the correct simulation for such differential equations; the latter, with delta = 1, is wholly misleading.

Advocates of Daisyworld as well as skeptics are similarly vulnerable to such errors. Even Lovelock's most cited article introducing Daisyworld [41], while qualitatively reasonably correct, nevertheless contained significant quantitative errors for related reasons, as pointed out in [24].

4.3 Rein Control: Stability without a Setpoint

The Daisyworld phenomenon is a form of homeostasis, since it involves stabilizing some essential variable (e.g., room temperature or body temperature) at or around some appropriate value. There are basically two possible classes of mechanisms for achieving this, illustrated in Figure 5.

Figure 5.

Different strategies for staying on the road: (1) move to centerline (setpoint), (2) stay away from edge.

Figure 5.

Different strategies for staying on the road: (1) move to centerline (setpoint), (2) stay away from edge.

The first—often wrongly assumed to be the only way—is the way central heating systems regulate room temperature T despite varying external temperature Te. We need the regulatory system to provide a correcting temperature offset Tc such that T = Te + Tc settles down to the desired Tsetpoint. The difference between the actual temperature T and the setpoint desired temperature is measured; depending on whether this is positive or negative, the system acts so as to reverse it:
$dTcdt=MTsetpoint−T.$
(7)
Here the function M( ) is a monotonically increasing function, in its simplest form a linear function M(x) = kx with k > 0; the size of k sets the time scale of the response to any perturbations. The system temperature would then move to T = Tsetpoint regardless of what value the external perturbation Te takes. More realistically, such a biological system will have limits beyond which the system fails to regulate efficiently. Nevertheless, the hallmark of this traditional class of feedback system is: Within such limits there is regulation to a fixed setpoint.

The second biological style of homeostasis is different and has no setpoint. Engineers typically are unaware of this.5 Clynes [11] was among the first to point out that biological control systems operate with internal control variables that can only take non-negative values—and hence, in order to control against both being too hot and being too cold, they must have two separate pathways for doing so. This is sometimes called the principle of unidirectional rate sensitivity; Clynes also called it rein control, a metaphor that is easier to understand. Since the reins of a horse can only pull, not push, in order to steer a horse in either direction you need a different rein on each side—as biological control systems in fact have.

The rein control version is to use two separate control variables, pulling against each other, so as to reach an equilibrium; a classic biological example would be antagonistic pairs of muscles. I illustrate this here with an example rein control system for staying away from both edges of a road (Figure 5(2)) to reach equilibrium somewhere in between. Let x represent the position on the road relative to the centerline at x = 0; positive x is on the right of the road. There is an external perturbing force, say the wind W, tending to push a vehicle (or horse) to one side or the other. Our rein control consists of one or two correcting reins, CL and CR (Figure 6). We use the same naming convention as for horses' reins, with the right rein CR being the one responding to being too far to the left (negative x); hence CR is only zero or positive, CL is only zero or negative.

Figure 6.

The heavy black line illustrates one possible example of a right-rein correctional force CR.The equilibrium value of CR is: 0 for x > −1 (“free rein”); 1 − x for −2 ≤ x ≤ −1 (“soft rein,” indicated by spring); and as large as is needed for x < −2 (“hard rein,” indicated by rope overriding spring). CR will be restrained from going to infinity by the circular causation of x being dependent on C, as well as C dependent on x: See equations in main text. Heavy gray line illustrates a similar possible left-rein CL. The inset plot shows final x-values for a rein control system using such reins CL, CR to correct against different perturbations W; see main text.

Figure 6.

The heavy black line illustrates one possible example of a right-rein correctional force CR.The equilibrium value of CR is: 0 for x > −1 (“free rein”); 1 − x for −2 ≤ x ≤ −1 (“soft rein,” indicated by spring); and as large as is needed for x < −2 (“hard rein,” indicated by rope overriding spring). CR will be restrained from going to infinity by the circular causation of x being dependent on C, as well as C dependent on x: See equations in main text. Heavy gray line illustrates a similar possible left-rein CL. The inset plot shows final x-values for a rein control system using such reins CL, CR to correct against different perturbations W; see main text.

The effect on x of these perturbations and correcting forces may be summarized as
$1τdxdt=W+CL+CR−x$
(8)
with an equilibrium at x = W + CL + CR. Different positive values for τ in the various differential equations will set different time scales for each. It would be important to take account of this if fully analyzing the dynamics, but for our present purposes of equilibrium analysis we may simplify by assuming all τ = 1.
The right rein CR, as illustrated in Figure 6, obeys
$dCRdt=max0−1−x−CRifx>−2.9a−2−xifx≤−29b$
The left rein CL, as shown, behaves similarly:
$dCLdt=min01−x−CLifx<2.10a2−xifx≥210b$

We append Matlab code for simulating these equations ((8), (9a, b), (10a, b)) numerically.6 The resulting plot (inset in Figure 6) shows that x is maintained within the hard boundaries [−2, 2], regardless of the W-value. But there is no setpoint; the final value of x is dependent on W. This example is just one from a large family of possible reins with any combination of hard and soft and free. Conventional negative feedback control to a setpoint can be seen as a specific restricted member of this family where both CL and CR have a hard rein fixed to one setpoint.

4.4 Why Call This Homeostasis?

What, then, justifies calling this a form of homeostasis, when the resulting equilibrium value of x will vary according to the perturbing external force W? The fact that, as W varies across a range of values, equilibrium x varies across a reduced range, less than in the absence of such reins. This can be seen from simple mathematical considerations, and (as an aside for chemists or economists familiar with this) it would appear to be a very basic example of Le Chatelier's principle [27] in action. This principle is usually stated as something like “the typical consequence of any perturbation to a system in equilibrium is a shift to a new equilibrium that partly counteracts the perturbation.” In many cases, such as Daisyworld, this counteraction to a perturbation results in viability being possible over a wider range of external perturbations than would otherwise have been the case. Even if one does not stay at the centerline, one still stays on the road.

We note that Clynes' rationale [11] for biological systems' use of such strategies was the observation that internal bodily control signals used variables, such as chemical concentrations, or the frequency or intensity of spikes, that could vary in positive values but not go below zero. Though artificial systems may not be so constrained, as an aside for those familiar with artificial neural networks it may be of interest that this relates directly to the rectified linear unit (ReLU) activation function f(x) = max(0, x) that has recently become popular. This basic nonlinearity is surprisingly powerful.

Our next example of circular causation is the homeostat (and, by extension, an autopoietic entity). Ashby's [2] motivation was to design a machine that learnt through experience to maintain essential variables within bounds. How can a kitten (or machine) learn to avoid the fire without prior knowledge of appropriate input-output responses to heat and pain?

Ashby's answer was to have a mechanism triggered by any crossing of the viability boundary that in essence caused some random variation of the input-output mapping (Figure 7(a)). Any inappropriate response would continue further variation, but if and when an appropriate input-output mapping was chanced upon, that formed a stable viable attractor to the circular dynamics. Conceptually this is similar to Darwinian random variation and selection, except within an individual rather than a population. Herein lies a problem.

Figure 7.

(a) Schematic of homeostat. Second feedback path triggers S if essential variables are outside limits. (b) Related schema for autopoiesis. (c) Models (a), (b) are part of my world.

Figure 7.

(a) Schematic of homeostat. Second feedback path triggers S if essential variables are outside limits. (b) Related schema for autopoiesis. (c) Models (a), (b) are part of my world.

5.1 When Do Homeostats or Autopoietic Entities Die?

In this context viability is a binary dead-or-alive distinction; a viability boundary is a definite line. But if random variation is only triggered by crossing such a line, surely that is too late: The homeostat is dead. So the viability signal that Ashby needs for the desired “ultrastability” is inherently paradoxical.

I believe Ashby made a tactical error; he fell into the same pre-Copernican trap. Viability is both a global binary property of an organism (here the homeostat) and a label for a local signal that triggers variation. To identify these global and local senses would be a category error. Here the local sense needs to be analogue not binary (e.g., anything correlated with life expectancy, associated with current essential variables) and probabilistically pulling the trigger for variational change. Even if both viabilities were step functions, they could not be the same step function.

This same issue of the viability boundary is inherited directly in autopoietic theories [37] (see Figure 7(b)) and Di Paolo [13] makes this explicit. Conservation or breakdown of organization in an autopoietic system is a binary step function, so how, Di Paolo asks, can that provide a gradient that represents the significance of the danger for the organism (e.g., kitten)? The claim appears to be that some such gradient (Di Paolo develops a concept of adaptivity to provide it) is needed for a pointer towards the danger lurking across the viability boundary. But it is a category error to identify direction in essential-variable space with direction in the room with the fire; as with drive and viability in the earlier examples, this is a misidentification of local with global meanings of a term. There should be some correlation between the two, but necessarily there cannot be identity between them.

One obvious class of candidate local signal would be any scalar entity that, at least crudely, correlated negatively with life expectancy. For a kitten approaching a fire, heat and (when getting too close) associated pain have just that crude negative correlation. With kittens, we assume that some such crude correlation has been promoted through Darwinian natural selection. But this is not the only explanation for such correlations—even natural rock arches subject to erosion (that clearly do not have origins in Darwinian evolution) tend to react to local rock falls (crudely negatively correlated with arch life expectancy) by rearrangement of forces in a manner (Le Chatelier's principle again; see Section 4.4) that promotes their survival. How? Through normal settlement.

6 Circular Causation and Normal Settlement

This article has attempted to show why circularly causal systems are often so baffling. The type of explanatory strategy proposed is based on what I like to dignify as Harvey's universal principles of normal settlement (NS)—the pompous title no doubt failing to conceal that these are just restatements of the blindingly obvious:

• (NS1) Any complex system will spend more time in some parts of its phase space than others; the more visited including attractors of all sorts, and stable and (over extended periods) metastable states.

• (NS2) If you find the system in one of the common regions, you should be less surprised than when you find it in a less common region.

If you agree that a defining factor (or, as I would claim, the defining factor) for an explanation is that it should reduce the degree of surprise in the observer by underwriting (implicit or explicit) provisional predictions, then it follows from NS principles that identifying the more common regions of phase space is a step in the right direction. Sometimes this step is bigger than others; if the system can be shown to have a single point attractor (and sufficient time to reach it), that is a stronger result than where there are multiple attractors with little evidence to distinguish between them.

Explanations presented on this basis are frequently by some standards incomplete. For example, they may fail to address the origins of a phenomenon, even that as simple as the stone arch example above. But nevertheless they can be predictive, and it is clear how they can be modified and improved whenever a new phenomenon surprises. Incidentally, this style of explanation lies at the root of Darwinian natural selection, a different NS that is a specialization of the more universal NS principles presented here. Natural selection = normal settlement + heredity.

Copernicus removed our firm foundation on earth at the center of the universe. Special relativity eliminated the fixed ether and required new frameworks for physics. Life and cognition are much more complex than physics, and still await their Copernican, relativistic revolutions. Even the simplest protoagent can confuse us with its unfamiliar circular causation, its lack of a settled foundation on which to build analysis.

It is suggested here that such confusion arises often through mistaking the map for the territory, careless identifying of local and global concepts as the same. GOFAI and even connectionism are full of such pitfalls [17], through attributing cognitive-agent-level functions to component parts. Sometimes this is useful conscious metaphor. Too often it is taken literally and people fall into the trap; so-called internal representations in the brain would be a classic example.

Autopoiesis takes circular causation most seriously. But even here there is yet further circularity, often unrecognized. Physics explains and redefines the “stuff” of our world, relates tables to atoms. But models of life and cognition are themselves stuff in the world that we live in; we ourselves are inside what we seek to understand (Figure 7(c)).

Models of biological or cognitive systems, as used in biology or artificial life, are typically conceived from an assumed “God's-eye” viewpoint outside the system of study. We need to carefully distinguish between what is perceptually available from our privileged external observer's viewpoint and what is perceptually available to the organisms or agents of study. A satirical novel (and early ALife speculation) that highlights this issue is Flatland [1, 21].

Flatland describes a plane 2D world in which people are simple geometrical polygons or lines, and their world and interactions with it follow the consequent 2D constraints. Whereas we, from our 3D perspective, can imagine a sphere passing through the plane, to its inhabitants this (or its intersection with their world) would appear as a dot out of nowhere, expanding into a circle of maximum diameter, which then dwindles and disappears. In turn, these 2D inhabitants are aware of further constraints that hold for a 1D “Lineland” or even a 0D “Pointland.” Visualizing such lesser dimensions is relatively easy, but for them visualizing their position within higher dimensions is too much of a challenge.

This novel, though now dated, nevertheless highlights a core issue when we try to understand other worlds and our own world. If we choose to assume (whether wisely or not) that ultimately the agents in that world and the objects they interact with are basically made of the same stuff, then consequences follow. For agents in that world, objects arise (or are perceived, or enacted) out of stuff-stuff interactions—and this cannot be the same as stuff. So, as a matter of logic, what is “matter” to an agent in that world cannot be the same as any stuff that both agents and objects are built from. Failure to make this distinction is the same kind of category error as the confusions between local and global meanings of a term, highlighted in earlier sections of this article.

This is circularity taken to the limit, but we need to face up to the fact that cognition is not just a phenomenon that inescapably involves circular causation—it is a circle that includes us ourselves within it (Figure 7(c)). The “stuff” in our models must be different from the “stuff” of our models.

6.2 Comparisons with Other Types of Explanation

Conventionally the background section of an article, citing other relevant work, appears at the beginning. By citing existing work in the field, we present the foundation on which new results can be built, in the tradition of linear causal explanations. For this article it seems more appropriate to place it here at the end, for two reasons. Firstly, there is surprisingly little previous work on circular causation to cite; we have already mentioned that cybernetics [41] and autopoiesis and enaction [3739], while discussing circular causation, do not focus on the particular confusions and misunderstandings we address here. Secondly and more importantly, it is only after developing the analysis above that we can compare and contrast with existing ideas that look related, but on inspection may turn out to be somewhat different.

The view of circular causality presented here clearly overlaps with discussions on dynamical systems approaches to cognition (e.g., [4, 36]), and with the approach associated with earlier work in evolutionary robotics (ER) (e.g., [5, 23]). The importance of attractors in such analysis was sometimes explicit [23]. Ashby [2] was an important early influence. It is commonplace within ER to view the task of the roboticist as that of shaping the behavior space of robots so as to have the desired attractors.

Some would relate the view presented here to notions of “emergence.” I reject that label, since it has been so widely used in such vague undefined terms as to be unhelpful. Where attempts have been made to define emergent explanations clearly [3], they have been negative definitions: Emergence has been defined in terms of filling a gap in the observers' expectations—this corresponds to my understanding of explanations as a response to surprise—but then there is rather little specified as just how this gap is being filled.

The view presented here should be seen as incompatible with some attempts [10] to subscribe to a dynamical systems or enactive approach to cognition that nevertheless retains cognitivist and representationalist explanations. This confuses representations as explananda (crucial features of our human cognition that need explaining) with representations as explanans (used as part of an explanation). It is an example of a basic category mistake confusing brains with minds or persons, while the view presented here is trying to make such distinctions very clear. The cognitivist carelessly elides the global sense of “representations,” as used by people in everyday life, with a local sense as used metaphorically for a homunculus in the brain [17].

There is an ongoing debate in the philosophy of science literature [25, 30] about the difference between (i) “mechanistic” explanations in biology and (ii) explanations that use “dynamical and mathematical models.” On first reading, there could be loose parallels here with what I have been calling explanations via (a) linear causation and (b) circular causation. But Silberstein and Chemero [32] see “localization and decomposition” of the component parts of explanations as a prime distinguishing factor between (i) what has “localization and decomposition” and (ii) what is not characterized thus. The arguments presented here distinguishing (a) linear from (b) circular explanations do not appeal to this factor.

There is an interesting connection between the ideas of circular causation presented here and repeated attempts (e.g., [12, 28]) to use notions of “information” as some crucial ingredient that explains why or how living organisms apparently cheat the second law of thermodynamics. Schrödinger [30] brought ideas of information into biology, specifically genetics, and the tools of information theory have been invaluable. But technical usefulness has been too often mixed with analytical carelessness, with category errors. It seems to me that these ideas rely on Shannon information, and, as Shannon [31] makes clear, his theory would have been better named communication theory than information theory. Shannon information is specifically a relational concept, relative to a sender and receiver communicating in the presence of noise. This places it squarely within the type of context in which circular causal explanations are appropriate, and principles of normal settlement are crucial. It also means one needs to distinguish carefully between local (information relative to component parts of a system conceived as metaphorical homunculi) and global (including information from an external “God's-eye” perspective) senses of information—and this is where too many fail and fall into exactly the same trap as the cognitivist's misuse of representations discussed above. In summary, these mistakes are symptoms of the proponents of information in this context failing to understand how explanations based on circular causation work.

7 Conclusions

Of course, circles do not have conclusions, but articles do, and we should summarize the main points to stress.

We define circular causation as what we appeal to in causal explanations of phenomena that rely on multiple contemporaneous interactions between component parts. Whereas linear explanations are built up incrementally, with each stage in principle fully explained and settled before the next stage is added, in contrast circularly causal explanations require the full circle to complete before they work.

When we refer to causality (circular or linear), we here really refer to the type of explanation (circular or linear) that we use. The role of an explanation is taken as allowing predictions and reducing surprise. “Circular reasoning” typically refers to a logical fallacy, and this may in part explain why the concept of circular causation is often treated with suspicion. But mainly people are just unfamiliar with the idea; remarkably little is written about it.

Circularly causal explanations typically appeal to principles of normal settlement. We identify regions of phase space of a system that are more likely (often overwhelmingly so) to be visited than the average. So often we are dealing with dynamical systems, and their attractors and metastable regions. We typically explain why if a system is in a specific state or in a specific attractor then it is likely to stay like that. Typically we may not be offering any explanation for why the system arrived in that state in the first place. Linear explanations are usually needed for origins.

Circularly causal phenomena provoke lots of confusion in people unfamiliar with them. Examples such as DDWFTTW and Daisyworld have many disbelievers, with many proofs offered that the phenomena cannot be real. Though convincing to some, these proofs are flawed.

One signature error that often accompanies this confusion is a misidentification of terms used locally to describe some component part of the system, with similar terms used globally to describe the global behavior of the system. Examples given above include the concept of drive in DDWFTTW, of viability (and synonyms) in Daisyworld [18], and of direction in Di Paolo's analysis of autopoiesis [13]. In neuroscience such confusion is common, and Bennett and Hacker [7] refer to this as the mereological fallacy. I prefer to call it a straightforward category error, which circular causation seems particularly prone to. The cognitivist confusion between representations that people use in their life and “representations in the brain” is a notorious example, and I have suggested above that information is likewise misinterpreted by some.

Biology and cognition—and thus necessarily artificial life—are domains that are permeated throughout by multiple overlapping feedback circuits, both within organisms and further through their coupling with the environment. It is incumbent on us to recognize the underlying principles needed for explanations, to be aware that our intuitions (too often based on linear assumptions) may be wrong, and to look out for the typical errors identified here. Ultimately, if we want to understand human cognition, we must be prepared for the consequences of being within the cognitive circle ourselves, not just outside observers.

Notes

1

For mention of this episode within an entertaining journalistic account of how Rick Cavallaro and colleagues had such difficulties with disbelievers, see the article in Wired magazine, 27 August 2010. https://www.wired.com/2010/08/ddwfttw/.

2

Cairns-Smith [9] used the same example of an arch for discussion of very similar issues. I thank two reviewers for pointing this out.

3

Though the Matlab examples shown in this article do not explicitly show such principled reality checks in the code, they were indeed done.

4

Basic demonstration (just one daisy type) of Daisyworld Matlab code:

5

A search of IEEE publications for “rein control,” via IEEE Xplore Digital Library, finds just one mention, namely, an article by the present author. Google Scholar offers 57 citations for Clynes' main article on this [11], none of them from a (non-biological) engineering context.

6

Basic demonstration of rein control Matlab code:

References

1
Abbott
,
E. A.
(
1884
).
Flatland: A romance of many dimensions
.
London
:
Seeley
.
2
Ashby
,
W. R.
(
1952
).
Design for a brain
.
London
:
Chapman and Hall
.
3
Baas
,
N. A.
, &
Emmeche
,
C.
(
1997
).
On emergence and explanation
.
Intellectica
,
2
(
25
),
67
83
.
4
Beer
,
R. D.
(
2000
).
Dynamical approaches to cognitive science
.
Trends in Cognitive Science
4
(
3
),
91
99
.
5
Beer
,
R.
, &
Gallagher
,
J. C.
(
1992
).
Evolving dynamic neural networks for adaptive behavior
.
,
1
(
1
),
91
122
.
6
Behe
,
M.
(
1996
).
Darwin's black box
.
New York
:
The Free Press
.
7
Bennett
,
M. R.
, &
Hacker
,
P. M. S.
(
2003
).
Philosophical foundations of neuroscience
.
Oxford, UK
:
Blackwell Publishing
.
8
Bethwaite
,
F.
(
2010
).
Faster sailing techniques
.
London
:
.
9
Cairns-Smith
,
A. G.
(
1985
).
Seven clues to the origin of life
.
Cambridge, UK
:
Cambridge University Press
.
10
Clark
,
A.
(
1998
).
Being there: Putting brain, body, and world together again
.
Cambridge, MA
:
MIT Press
.
11
Clynes
,
M.
(
1969
).
Cybernetic implications of rein control in perceptual and conceptual organization
.
Annals of the New York Academy of Sciences
,
156
,
629
670
.
12
Davies
,
P.
(
2019
).
The demon in the machine
.
London
:
Allen Lane
.
13
Di Paolo
,
E. A.
(
2005
).
.
Phenomenology and the Cognitive Sciences
,
4
,
429
452
.
14
Ford
,
M.
(
2018
).
Architects of intelligence
.
Birmingham, UK
:
Packt
.
15
Gabor
,
D.
(
1949
).
Microscopy by reconstructed wavefronts
.
Proceedings of the Royal Society A
,
7 July
,
454
487
.
16
Gaunaa
,
M.
,
Øye
,
S.
, &
Mikkelsen
,
R.
(
2009
).
Theory and design of flow driven vehicles using rotors for energy conversion
. In
Proceedings of EWEC 2009, Marseille
.
17
Harvey
,
I.
(
1996
).
Untimed and misrepresented: Connectionism and the computer metaphor
.
AISB Quarterly
,
96
,
20
27
.
18
Harvey
,
I.
(
2015
).
The circular logic of Gaia: Fragility and fallacies, regulation and proofs
. In
P.
Andrews
,
L.
Caves
,
R.
Doursat
,
S.
Hickinbotham
,
F.
Polack
,
S.
Stepney
,
T.
Taylor
, &
J.
Timmis
(Eds.),
Proceedings of the European Conference on Artificial Life 2015
(pp.
90
97
).
Cambridge, MA
:
MIT Press
.
19
Harvey
,
I.
(
2016
).
Social systems and ecosystems: History matters
. In
C.
Gershenson
,
T.
Froese
,
J. M.
Siqueiros-García
,
W.
Aguilar
,
E.
Izquierdo
, &
H.
Sayama
(Eds.),
Proceedings of Artificial Life Conference 2016
(pp.
418
425
).
Cambridge, MA
:
MIT Press
.
20
Harvey
,
I.
(
2017
).
Going round in circles
. In
C.
Knibbe
et al
(Eds.),
ECAL2017 Proceedings of the European Conference on Artificial Life
2017
(pp.
198
199
).
Cambridge, MA
:
MIT Press
.
21
Hinton
,
C. H.
(
1907
).
An episode on Flatland: Or how a plane folk discovered the third dimension
.
London
:
Swan Sonnenschein
.
22
Hinton
,
G. E.
, &
Anderson
,
J. A.
(
1981
).
Parallel models of associative memory
.
Hillsdale, NJ
:
Erlbaum
.
23
Husbands
,
P.
,
Harvey
,
I.
, &
Cliff
,
D.
(
1995
).
Circle in the round: State space attractors for evolved sighted robots
.
Robotics and Autonomous Systems
,
15
,
83
106
.
24
Isakari
,
S. M.
, &
Somerville
,
R. C. J.
(
1989
).
Accurate numerical solutions for Daisyworld
.
Tellus
,
41B
,
478
482
.
25
Kaplan
,
D.
, &
Bechtel
,
W.
(
2011
).
Dynamical models: An alternative or complement to mechanistic explanations?
Topics in Cognitive Science
,
3
,
438
444
.
26
Kirchner
,
J. W.
(
1989
).
The Gaia hypothesis: Can it be tested?
Reviews of Geophysics
,
27
,
223
235
.
27
Le Chatelier
,
H
. (
1884
).
Sur un enoncé général des lois des équilibres chimiques
.
Comptes Rendus de l'Académie des Sciences
,
99
,
786
789
.
28
Maynard Smith
,
J.
, &
Szathmáry
,
E.
(
1995
).
The major transitions of evolution
.
Oxford, UK
:
Oxford University Press
.
29
McCulloch
,
W. S.
(
1960
).
What is a number, that a man may know it, and a man, that he may know a number?
General Semantics Bulletin
,
26/27
,
7
18
.
30
Schrödinger
,
E.
(
1944
).
What is life? The physical aspect of the living cell
.
Cambridge, UK
:
Cambridge University Press
.
31
Shannon
,
C. E.
(
1948
).
A mathematical theory of communication
.
Bell System Technical Journal
,
27
(
4
),
623
666
.
32
Silberstein
,
M.
, &
Chemero
,
A.
(
2013
).
Constraints on localization and decomposition as explanatory strategies in the biological sciences
.
Philosophy of Science
,
80
(
5
),
958
970
.
33
Thomas
R.
, &
D'Ari
,
R.
(
1990
).
Biological feedback
.
Boca Raton, FL
:
CRC Press
.
34
Thomas
,
R. L.
, &
Kaufman
,
R.
(
2001
).
Multistationarity, the basis of cell differentiation and memory. I. Structural conditions of multistationarity and other nontrivial behavior
.
Chaos
,
11
(
1
),
170
179
.
35
Thomas
,
R. L.
, &
Kaufman
,
R.
(
2001
).
Multistationarity, the basis of cell differentiation and memory. II. Logical analysis of regulatory networks in terms of feedback circuits
.
Chaos
,
11
(
1
),
180
195
.
36
Van Gelder
,
T
. (
1998
).
The dynamical hypothesis in cognitive science
.
Behavioral and Brain Sciences
,
21
(
5
),
615
628
.
37
Varela
,
F. J.
,
Maturana
,
H. R.
, &
Uribe
,
R.
(
1974
).
Autopoiesis: The organization of living systems, its characterization and a model
.
BioSystems
,
5
,
187
196
.
38
Varela
,
F. J.
(
1984
).
The creative circle: Sketches on the natural history of circularity
. In
P.
Watzlawick
(Ed.),
The invented reality
(pp.
309
324
).
New York
:
W. W. Norton
.
39
Varela
,
F. J.
,
Thompson
,
E.
, &
Rosch
,
E.
(
1991
).
The embodied mind: Cognitive science and human experience
.
Cambridge, MA
:
MIT Press
.
40
von Foerster
,
H
. (
1979
).
Cybernetics of cybernetics
. In
K.
Krippendorff
(Ed.),
Communication and control
(pp.
5
8
).
New York
:
Gordon and Breach
.
41
Watson
,
A. J.
, &
Lovelock
,
J. E.
(
1983
).
Biological homeostasis of the global environment: The parable of Daisyworld
.
Tellus
,
35B
,
284
289
.