A Neural Model for Insect Steering Applied to Olfaction and Path Integration

Insects display an astonishing range of behaviors that include highly directed movements. For example, male moths navigate toward females emitting pheromones (Marsh, Kennedy, & Ludlow, 1978; Obara, 1979) and female crickets move toward singing males (Simmons, 1988; Balakrishnan & Pollack, 1996). Other insects use visual cues to maintain a straight heading over short or long distances (dung beetle: Dacke, Nordström, & Scholtz, 2003; Baird, Byrne, Smolka, Warrant, & Dacke, 2012; monarch butterfly: Mouritsen & Frost, 2002; bogong moth: Dreyer et al., 2018) and can even rely purely on memory to navigate home (Srinivasan, 2015; Honkanen, Adden, da Silva Freitas, & Heinze, 2019). While the cues used for navigation are different in these examples, they elicit very similar behaviors: upon encountering an appropriate stimulus, the animal chooses a direction with respect to that stimulus and begins moving in that direction. If the stimulus is temporarily lost, searching behavior is initiated. Thus, the motor patterns elicited by different kinds of stimuli can be remarkably similar.

The answer likely lies in the lateral accessory lobes (LAL), a paired neuropil flanking the CX. The LAL forms part of the pheromone-processing pathway that has been described in detail in the silkworm moth (Namiki, Iwabuchi, Pansopha Kono, & Kanzaki, 2014; Kymre et al., 2021). While the CX integrates information from modalities such as vision and mechanoreception, it does not appear to process olfactory information. Odors, including pheromones, are integrated in higher-order neuropils including the lateral horn, mushroom bodies, and finally the LAL. The LAL has been described as a premotor center, as several types of descending neurons that project to thoracic motor centers have postsynaptic endings in the LAL (Namiki, Dickinson, Wong, Korff & Card, 2018). It has been suggested that interactions between the two LALs are sufficient to mediate search behaviors triggered by, for example, pheromone input, but that goal-oriented behaviors require CX input into the LAL network (Namiki & Kanzaki, 2016; Álvarez-Salvado et al., 2018). Indeed, CX output neurons project to the LALs, but how they interact with the LAL circuitry is currently unknown. Additionally, how descending neurons encode motor commands on a population level is currently not well understood, although multiple recent studies have been able to dissect single neural circuits that underlie specific behaviors (Schnell, Ros, & Dickinson, 2017; Cande et al., 2018; Namiki, Wada, & Kanzaki, 2018).

One such behavior, which has been examined in detail, is the pheromone-following behavior of silkworm moths. Male silkworm moths display a highly stereotyped behavioral sequence when following a female's pheromone plume. Upon first contact with the plume, the moth responds with a “surge,” that is, a straight movement toward the source of the odor. When the odor plume is lost, several ways of reacquiring the plume have been described; most notably casting, during which the moths walks in a zig-zag pattern until it finds a new odor pocket, and looping downwind (Cardé & Willis, 2008). Early studies have identified several descending neuron types whose activity correlates with turning behavior when a male moth orients in a pheromone plume (Mishima & Kanzaki, 1999). Among these, the most notable are “flip-flop” neurons, which are bistable neurons that switch between a high-activity and a low-activity state in response to a trigger stimulus (Olberg, 1983; Kanzaki, Ikeda, & Shibuya, 1994; Mishima & Kanzaki, 1999). That is, the same stimulus can cause the neuron to increase or decrease its firing, depending on whether it is in the low or high activity state, respectively, when that stimulus occurs.

These neurons have postsynaptic terminals in the LALs, and their axons descend through the ventral nerve cord and synapse onto neck motor neurons, which in turn activate neck muscles that control head movements (Kanzaki & Mishima, 1996; Mishima & Kanzaki, 1998). Thus, if the left-descending flip-flop neuron is in its high-activity state, the left neck motor neuron and the left neck muscle are also active, causing the head and consequently the moth to turn left. Although this network has been described in the context of pheromone following, other studies have shown that flip-flop neurons can also be triggered by light flashes (Olberg, 1983) and sound (Zorovic & Hedwig, 2011). It therefore seems likely that flip-flop neuron-mediated steering may constitute a general form of targeted steering, independent of the stimulus modality that drives the behavior (Steinbeck, Adden, & Graham, 2020).

In this study, we aim to evaluate whether a basic flip-flop neuron network can produce naturalistic steering in a simulated odor plume and when presented with compass input via a neural model of the central complex. To this end, we present a rate-based and a spiking computational model of a simple flip-flop network. The two models presented here allow us to compare two different implementations of the same network, where both models follow the same connectivity pattern. The rate-based model uses continuous-valued sigmoid neurons, while the spiking model uses leaky integrate-and-fire (LIF) spiking neurons. Both models are effective at navigating in the simulated olfactory and visual tasks; however, the spiking model produces more realistic trajectories. Comparing the behavior of our models to behavioral data from male silkworm moths, we find that this simple flip-flop-based neural circuit is sufficient to replicate the basic characteristics of the moths' paths. Furthermore, we describe looping behavior as an emerging orientation strategy when sensory input is directionally ambiguous. Finally, we demonstrate for the first time that the flip-flop network can work as a general steering network when combined with a computational model of the CX (Stone et al., 2017). This study is therefore a step toward closing the gap between higher processing centers in the brain that make navigational decisions, such as the CX, and the thoracic motor circuits that ultimately move the insect.

The neurons modeled here were physiologically and anatomically described (Olberg, 1983; Kanzaki, Sugi, & Shibuya, 1992; Mishima & Kanzaki, 1999; Kanzaki et al., 1994). We infer input and output regions of neurons from their anatomical appearance, that is, smooth terminals are assumed to be inputs, while varicose terminals are assumed to be outputs. Two neurons are assumed to be connected if the input region of one neuron overlaps with the second neuron's output region. The model connections are furthermore based on the network proposed in Mishima and Kanzaki (1999) with some small modifications.

The network consists of two pairs of neurons: one flip-flop neuron (FF) and one protocerebral bilateral neuron (PBN; see Figure 1A) per hemisphere. Both cell types receive input directly from the plume or from the output neurons of the central complex (CPU1/PFL neurons) when connected to the path integration network (PI). PBN neurons were proposed to provide bilateral inhibition between the two LALs (Mishima & Kanzaki, 1999; Kanzaki, Nagasawa, & Shimoyama, 2005) and are therefore modeled to inhibit the contralateral FF neuron. FF neurons have excitatory connections directly onto the contralateral motor, based on the finding that FF neuron activity correlates with neck motor neuron activity (Kanzaki & Mishima, 1996).

We present two implementations of this network, which we call the rate-based model and the spiking model. The rate-based model uses continuous-valued sigmoid neurons, the most common neural model used in insect brain network models. However, while such neurons are easier to work with and build models from, they abstract away many of the underlying biological details. To complement this, we also present a spiking model built from leaky integrate-and-fire (LIF) neurons, bringing richer temporal dynamics to the model. While this is still abstracted, we show that the spiking model leads to more realistic behavior.

The general approach for creating the spiking model is known as the neural engineering framework (NEF; Eliasmith & Anderson, 2003). This allows us to take any particular desired dynamics (such as the dynamics of a flip-flop neuron) and construct it using more basic components (such as a group of spiking LIF neurons). In particular, here we approximate one FF neuron using 100 recurrently connected spiking LIF neurons. The NEF treats this as an optimization problem and finds the ideal connection weights among those components, such that the overall system produces behavior that is as close as possible to the desired rate-based description of an FF neuron. Our interpretation of the model is that this is one flip-flop neuron with 100 internal components, and the flip-flop behavior arises out of these internal interactions.

Both models are described in detail below. The source code is available at https://github.com/stanleyheinze/insect_steering. The spiking model was implemented using the software toolkit Nengo (Bekolay et al., 2014).

For the FF neurons, we added a mechanism to produce flip-flop behavior, where an input stimulus will switch a neuron from a high state to a low state and vice versa, depending on the state of the neuron preceding the stimulus. Since sigmoid neurons by themselves are too simple to produce this behavior, we added a feature to the rate-based model where if the current output $r$ is large (⁠$>$0.8) and the input is large (⁠$>$0.5), then the output of this FF neuron is reduced by 0.5 and the opposite FF is increased by 0.5. This produces the required flip-flop behavior (see Figure S1), but does not postulate a plausible mechanism whereby this behavior is produced. We present a more realistic mechanism in the next section on the spiking model.

This differential equation will give similar behavior to the sigmoid FF neuron described above. In particular, note that if a differential equation is outputting a value of the form $T-x$⁠, then the value $x$ will move toward $T$ until $T=x$⁠. This means that in the first and last cases above, the state of the FF neuron will go to 1 (i.e., the FF is high), and in the second and third cases, the FF neuron will go to 0 (low). This gives us a differential equation approximation of the sigmoid FF.

The next step is to train a feedforward single-hidden-layer neural network that approximates $y=τf(x,u)+x$⁠. Importantly, we use LIF neurons in the hidden layer, but we have no nonlinearity at all in the input and output layers. These hidden layer neurons will eventually become the recurrently connected internal components that will approximate a FF neuron. To train this network, the training data are a set of randomly generated $x$ values and the $y$ values are generated using equation 3.4. The $τ$ parameter is the time constant of the postsynaptic currents that will be used in the next step, which we choose here to be 100 ms. While any neural network training approach can be used (e.g., backpropagation of error), here we take the simple approach of randomly generating the first layer of weights and using regularized least-squares minimization (i.e., ridge regression) to find the second layer of weights. This is a fast approach that works for any neuron model and avoids the difficulties of applying backpropagation to spiking neurons (Eliasmith & Anderson, 2003).

Finally, converting back to the time domain, we get $dxdt=f(x,u)$⁠. That is, this recurrently connected set of LIF neurons will approximate the desired flip-flop dynamics. Furthermore, since our feedforward network has no nonlinearity at the input or output, we can convert the system into a single pool of 100 LIF neurons whose recurrent connection weights are given by $W=WoutWin$⁠, where $Win$ is the weights from the input to the hidden layer and $Wout$ is the weights from the hidden layer to the output.

This overall method (Eliasmith & Anderson, 2003) lets us take any desired dynamics (written as a differential equation) and convert it into a pool of basic components (here we use 100 LIF neurons) that are connected to each other using exponential synapses. As the number of components increases, the accuracy of this approximation will improve, up to the limit of how well the feedforward neural network approximates $y=τf(x,u)+x$⁠. We can think of the resulting system as defining a spiking attractor network whose dynamics are governed by that function.

Note that we are not suggesting the flip-flop neurons must have recurrent connections, but rather that these neurons have a sufficiently complex internal state that such feedback is necessary to capture their properties. The resulting spiking model produces a notably different time course of response compared to the rate-based model, although the key qualitative characteristics are maintained. One crucial difference between the rate-based and spiking models is in how synchronization between the two FF cells is achieved. In the rate-based model, we have explicitly added a rule that forces the contralateral flip-flip to its up-state when an input pushes the ipsilateral flip-flop to the down-state, and vice versa. This is not necessary in the spiking model, as the internal components of a FF neuron can successfully switch the activity state of the neuron. However, there is no way for the internal components of one FF neuron to directly affect the contralateral FF neuron. There is, however, an inhibitory connection between the PBN and the contralateral FF, so this allows the system to push the contralateral FF to a low state when the ipsilateral FF and PBN get input from the sensor. This allows for some asynchrony to emerge in the spiking model rather than forcing it as in the rate-based model. Note that this type of asynchronous flip-flopping behavior can be most easily observed when using artificial input (see Figure S1A). It becomes less obvious when the agent is exposed to a naturalistic stimulus, such as the simulated odor plume (see Figure 1). In this more complex scenario, it is evident that the spiking model is only an approximation of the ideal equations given, and as such it is possible for both FFs to be high or low at the same time.

Ando et al. (2013) presented a robot that was steered through an odor plume by an on-board moth walking on a trackball. The moth's movement on the trackball was translated into wheel speeds for the robot. The authors presented trajectories for both the moth-controlled robot and the moth only in the odor plume. Here, we adjusted the parameters of this model such that the agent's trajectories were similar to the moths' trajectories presented in Ando et al. (2013), based not only on the trajectories but also quantitatively on the turn duration, turn angle, and turn velocity.

Using their robot, Ando and colleagues performed further experiments where they added a bias to the turning of the robot. That is, a constant signal was added to the left (or right) wheel while the moth was controlling the robot. This causes the moth to drift to the edge of the pheromone plume, but they show the moth is able to compensate for this and continue to follow the plume. We performed the same experiment in our simulation by adding a constant bias (in rad/s) to the moth rotation, causing the simulated agent to have an extra tendency to turn in a particular direction. Ando and colleagues show that moths' plume-following behavior is robust to this sort of manipulation and that the resulting paths tend to follow along the edge of the plume.

Behavioral measurement thresholds were defined in accordance with Ando et al. (2013) to allow for comparing the models to data from male silkworm moths. A turn was identified if the agent's turn duration was larger than 0.5 s, the turn angular velocity was larger than 5 deg/s, and the turn angle was larger than 30$∘$ (see Figures 2B and 2C; Ando et al., 2013). Loops were detected based on their high rotation rate, above 30 deg/s for at least 5 s. An experiment was considered successful if the agent arrived within 5 cm of the goal. All data were analyzed in Python 3.5.5.

In the plume experiments, the value of each sensor was between 0 and 1, so that the absolute difference between the two sensors could also fall between 0 and 1. However, the output from the path integrator had a narrower spread (0.5 to 1). While our models worked with that smaller difference, we also rescaled the path integrator output to a scale of 0 to 1 to test whether this would improve the models' behaviors.

To evaluate the behavior in this situation, we used exactly the same simulator as before (pompy) but removed the odor plume. We then caused the agent to take a random exploratory path by setting its rotation rate to be a gaussian white noise process with $σ=0.1$ rad/s while moving forward at a constant speed (see Table 1), for 15 seconds in total. The agent then attempted to return directly home using the CX output combined with our steering model. The simulation was continued for another 40 seconds. An experiment was considered successful if the agent arrived within 5 cm of the starting location.

In order to determine how realistic the behavior of our model is, we compared the simulated tracks quantitatively to data from silkworm moths (Bombyx mori, originally published in Ando et al. 2013; see Figure 3). The total turn duration of the rate-based model falls within the standard deviation of real moth data but was slightly higher for the spiking model (see Figure 3A). The total turn angle of both models fell within the standard deviation of real moth data (see Figure 3C), while the mean turn velocity of both models was 10% to 15% lower compared to real moths (see Figure 3B). However, both models performed well with respect to finding the origin of the plume, with a success probability of 0.84 to 1.0 (see Figure 3D). The tracks of both models display cross-wind zigzagging and straight surges, as well as loops (see Figures 3E and 3F), with the rate-based model having straighter paths than the spiking model. Overall, when comparing the rate-based and spiking models to moth data, we find that both models replicate real moth data reasonably well.

A further way of testing how well our models replicate real moth behavior was to add a turn bias to the simulation. When given a turn bias, silkworm moths were shown to track the edges of the odor plume instead of the center (Ando et al., 2013). This was also the case for our models (see Figure 4). When analyzing the angle between the current position of the agent and the source of the odor plume, we found that the models shift away from the center of the plume already at a turn bias of 1 rad/s (see Figures 4A and 4B). With increasing turn bias, the models' success rates decrease, and at a turn bias of 3 rad/s, no agent simulated by the rate-based model reaches the goal. The spiking model is more robust but starts failing at a turn bias of 5 rad/s (data not shown). While these results are more difficult to compare quantitatively to real moth data owing to the different ways of implementing the turn bias, it is clear that qualitatively, the models behave similar to real moths, as they track the edge of the plume rather than the center when given a turn bias.

Interestingly, looping behavior emerges from the spiking model but is only rarely observed in the rate-based model (see Figures 5A and 5B). In fact, we were unable to find ways of modifying the rate-based model to create these tight loops while still being able to successfully follow the plume (see Figures S3 and S4). It should be noted that while it is possible to observe clean flip-flop behavior in ideal circumstances (see Figure S1A), with more realistic input, the model exhibits a much richer array of behaviors (see Figure S1B), including times where both flip-flops are active and times when neither is active. This is due to the fact that the spiking model uses internal components to approximate the ideal flip-flop algorithm, but this approximation is imperfect. Thus, the spiking model may be able to get into a transient stable state where both flip-flops are on but one is slightly stronger, causing a long turn in one direction. Alternatively, there may be other low-level differences between the components used for the two sides of the model, leading to subtle asymmetries in the model. With this in mind, further examination of the looping behavior is warranted. Since moths have been described to perform loops when they lose the odor plume (Baker & Haynes, 1996), we analyzed the sensor values during looping. Loops were detected automatically based on their high rotation rates, allowing us to divide the trajectory into looping and nonlooping stretches (see Figure 5A). Analyzing the difference in value between the two sensors during looping, we find that looping occurs proportionally more often when the two sensors have similar or equal values (see Figures 5C and 5E), suggesting that looping emerges from the model when the sensor data are nondirectional. Furthermore, the average difference in sensor values is consistently lower during a loop than during a turn, and both terminate when the difference between left and right sensor increases (see Figure 5C). During a loop, FF neurons switch activity state less, and their output tends to remain stable on one side as compared to a turn (see Figure 5E).

To investigate what a possible advantage of looping could be, we compared the spiking FF network to a control agent in which the sensors were connected directly to the motor and showed no looping behavior. We removed the tendency to turn upwind in the absence of sensory input to simulate a situation in which the agent solely relies on the odor plume, and we tested both agents in successively sparser plumes. Under these conditions, the spiking FF agent had a significantly higher success rate than the control brain (see Figure 5F). Surprisingly, at very low plume concentrations, the control brain had a high success rate. We suspected that this may be a side effect of the agent pointing toward the odor source at the start of the simulation and simply walking straight in the absence of sensor input. Indeed, when we changed the starting angle such that the agent faced 90 degrees to the right of the odor source, the success rate of the control brain was reduced to almost 0, while the spiking FF agent remained successful at finding the source of the plume. This indicates that looping is indeed an orientation strategy that in the absence of all other sensory information can aid in locating the target.

Having established that the simple flip-flop networks are able to reliably replicate several characteristics of male silkworm moth behavior, we proceeded to test whether the models could also take CX output as input signals. For this experiment, we used a computational model of the CX that computes path integration in an anatomically constrained network (see Figure 6A; Stone et al., 2017). In short, path integration is a computation that combines, at each time step, the current heading of the animal (represented in TB1/delta7 neurons; see Figure 6A) with its forward speed (represented in TN/LNO neurons; see Figure 6A). The resulting vector (updated in the CPU4/PFN memory loop; see Figure 6A) points in the direction of the path's origin, and the length of the vector represents the distance of the animal from that origin. Thus, an animal that continuously updates this vector during an outbound path has the possibility to return to its origin in a straight line. Once the animal decides to return, the desired heading (encoded in the CPU4/PFN vector) is compared to the current heading (TB1/delta7 neurons), and mismatches between the two are transferred to an unspecified motor via CPU1/PFL neurons. As CPU1 neurons project from the CX to the LAL, it is plausible that they interact with flip-flop neurons there. Importantly, steering signals are encoded as an imbalance between the summed activity of all CPU1 neurons in the right hemisphere and those in the left hemisphere. We therefore used the summed CPU1 activity as input to the flip-flop network. Using the same model parameters as for odor-plume experiments, both the rate-based and the spiking model steer an agent back to its origin based on path integrator output (see Figure 6B), albeit with a lower success rate than the ideal path integrator (see Figure 6C). We assessed the accuracy of homing by analyzing the orientation of the agent relative to the origin, where 0$∘$ indicates that the agent is perfectly tracking along the straight line between the end of the outbound path and its origin (see Figure 6D). Without any connections to the steering model, the ideal path integrator peaks at an orientation of 0$∘$ and has a standard deviation of 15.4$∘$⁠. The spiking model also peaks at 0$∘$ but has a wider standard deviation of 67.9$∘$⁠. Interestingly, the rate-based model has one peak at 0$∘$⁠, a standard deviation of 50.2$∘$⁠, and two additional peaks at $±90∘$⁠.

Due to this odd distribution of orientations and the relatively low success rate for the two models (0.7 and 0.57, respectively), we examined different ways of connecting the path integrator to our steering system. There is no a priori reason for expecting an output value of 0.5 from the path integrator to mean the same thing as a 0.5 from the odor detection system. However, we do not want to postulate complex neural mechanisms between these two systems. Two simple things to adjust are the gain of this connection (which would correspond to increasing the number of synapses or moving the synapse closer to the spike initiation zone) and the bias current (which would correspond to changing the threshold at which the neuron will fire). While neither of these on their own significantly improved behavior, we found that adjusting both gain and bias such that a path integrator value of 0.5 is mapped to a 0 input to the steering system and a value of 1.0 stays at 1.0 (and intermediate values are linearly interpolated between these) greatly improved performance while keeping the same qualitative effects (see Figure 6). When rescaling the path integrator output to a scale between 0 and 1, we find that both models are significantly more successful as well as more accurate in tracking along a straight line back to the origin (see Figures 6C to 6F). At a sensor difference of 0.5 or above, the rate-based model's percentage of successful runs increases from 0.7 to 0.78, and the spiking model increases from 0.57 to 0.92 (see Figure 6C). Thus, the flip-flop circuit can be used to steer using multiple input sources, with minimal modifications.

We modeled a simple flip-flop network based on neurons that have been described in detail in the silkworm moth. The computational model has only two pairs of neurons: the flip-flop neurons (FF), which are bistable neurons, and the PBNs, which provide inhibition between the two flip-flops and thereby synchronize the two hemispheres. Note that in our model, there are no other connections between the two hemispheres; thus, this synchronization is entirely mediated by the PBNs. This network was modeled as both a rate-based model and a spiking model. Surprisingly, both models were able to replicate the behavior of real moths in an odor plume reliably, despite their simplicity. We found that additional neurons were not necessary to produce the behavior presented in this article (see Figure S2).

We also tested whether this simple steering network could serve as an interface between the CX and downstream motor centers by combining it with the CX path integrator network (Stone et al., 2017). We could show that both the rate-based and the spiking model can take input from the path integrator and use it for steering toward a target and that the efficiency of steering depends on the scaling of the input into the system.

We next discuss our findings with respect to the models, their behaviors, and the predictions and conclusions we can draw from these experiments.

When comparing the rate-based and the spiking models, both produced similar overall plume-following behaviors despite the difference in complexity. This reflects that rate-based models are, to a certain degree, a valid way of modeling neural networks, despite the many simplifications they involve. However, the more subtle behaviors of the system seem to be more realistic in the spiking version, in particular for looping behavior and for the influence of turning bias. Interestingly, when a turning bias is introduced (see Figure 4B), the spiking model produces the same edge-following behavior observed in the original experiment (Ando et al., 2013), while the rate-based model exhibits some shift but with a wider spread. Overall, this indicates that the spiking model produces more realistic behavior.

A major open question is how flip-flopping can be achieved in a real neuron. In the spiking model presented here, we combine low-level components, which consist of voltage buildup and spikes, to approximate the flip-flop behavior. While this produces more realistic behavior than the rate-based approach, more details could be added. In particular, various models of neural bistability exist (Camperi & Wang, 1998; Gruber, Solla, Surmeier, & Houk, 2003) that might serve as the basis for a more accurate model of flip-flop neurons.

When evaluating the behavior of both models, the turn angle, turn angular velocity, and turn duration agree well with values reported for male silkworm moths and fall within the mean $±$ 1 standard deviation of moths (Ando et al., 2013). The high positive deviation of the spiking model's turn duration (see Figure 3A) can be explained by this model's tendency to loop, leading to more and longer turns. The rate-based model also produced looping behavior but at a much lower rate than the spiking model. Loops occurred when the difference between the two sensors' values was very low and the directional information of the sensory input was therefore ambiguous. We could show that looping conferred a distinct advantage in sparser odor plumes, leading to significantly higher success rates as compared to a nonlooping control agent. This suggests that looping is a basic orientation strategy that allows an insect to sample its entire local environment and may be especially useful if the insect orients relative to an external cue or toward the source of an intermittent cue such as an odor plume. As an orientation strategy, looping has been described in several insects; for example, fruit moths perform a loop as a search mechanism when they lose the odor plume (Baker & Haynes, 1996); dung beetles perform a circular dance on their dung ball to take a “snapshot” of skylight cues before rolling their dung ball in a straight line relative to these cues (Baird et al., 2012; el Jundi, Foster, Khaldy, Byrne, Dacke, & Baird, 2016) and perform another dance after losing their bearing; and desert ants use a similar strategy to learn their visual surroundings during learning walks (Fleischmann, Grob, Wehner, & Rössler, 2017). Looping therefore appears to be a robust strategy to sample and learn the local sensory environment, as well as to reacquire a sensory signal that has been lost. Note that this orientation strategy would not be expected in simple taxis behaviors, in which the animal navigates purposefully toward or away from a sensory cue. In this situation, a very small difference between the two sensor values would be expected to elicit a straight walk, with the aim of keeping the difference as small as possible.

In addition to steering toward the source of an odor plume, we have shown that the models also steer well when getting input from the CX path integrator (PI) network published in Stone et al. (2017). The output from this network is in essence a steering signal that represents the difference between the intended heading and the current heading. This signal is asymmetric between the right and left hemisphere, depending on whether the agent needs to correct to the right or the left.

Our steering models can take this input and steer the agent toward its point of origin, using the same parameters that were used for odor-based steering. This was surprising, considering that the sensory input experienced in an odor plume is quite different from the input provided by the PI. Odor plume input is intermittent and varies at a high temporal frequency, whereas PI output is constant, changes smoothly without sudden jumps, and ideally varies within a relatively small range around the intended heading. Rescaling the PI output to match the range given by the odor plume resulted in successful control behavior without necessitating fundamental changes to the model. Biologically, this rescaling could be achieved by the presence of interneurons or neuromodulators or both, in the inputs to the flip-flop system and facilitate adapting to new sensory environments.

Our results suggest that the flip-flop system can take and integrate input signals from multiple modalities, including the CX, to generate downstream steering commands. One important limitation is that the multimodal input signals need to be directional, that is, there must be an imbalance between the signals in the right and left hemispheres. Here, we tested input signals that are derived from visual cues, but there is no reason to assume that the input should be restricted to olfactory and visual cues alone (Steinbeck et al., 2020). Silkworm moth flip-flop neurons are known to switch state in response to odor cues as well as light flashes, and bistable neurons of a similar morphology were found to switch state in response to auditory input in the cricket (Zorovic & Hedwig, 2011). Our data therefore support the idea that this neural network may work as a general-purpose steering network at least in the context of targeted orientation behaviors (excluding taxis behaviors).

One important question is whether the flip-flop neurons convey any actual advantage to the steering circuit. Indeed, the path integrator model without a flip-flop steering component works perfectly well (Stone et al., 2017; see also the “ideal” model in Figure 6). Similarly, pheromone plumes could be followed by directly connecting the left and right sensor data to downstream motor neurons, bypassing the flip-flop network altogether (see Figures 5F and S3). However, we have shown that one advantage of using the flip-flop network when following an odor plume is the emergence of looping behaviors, which increase the robustness of the behavior to disturbances such as intermittent loss of sensory input. Furthermore, the flip-flop models produce paths that cover a wider part of the available space while still successfully finding the plume source. We thus conclude that one purpose of the flip-flop system is to support reliable goal-directed behaviors while also facilitating exploratory behaviors that cause the insect to vary its position and orientation rather than following a direct path.

Our analysis generates several testable predictions. First and foremost, CPU1 neurons that project from the CX to the LAL are expected to have either direct or indirect excitatory synaptic connections with flip-flop neurons. To our knowledge, only two similar connections have been described so far. In the fruit fly, CPU1 neurons (PFL in Drosophila nomenclature) were shown to synapse onto bilateral LAL interneurons (Franconville, Beron, & Jayaraman, 2018; see also Hulse et al., 2021), as well as the ipsilaterally descending LAL neuron DNa02 (Rayshubskiy et al., 2020). However, whether CX output neurons also project onto contralaterally descending neurons in the LAL remains unknown. Finding the interaction sites between CX output neurons and LAL descending neurons, such as the flip-flop neurons modeled here, will be an important step toward understanding how the CX controls behavior.

Second, our findings support that the flip-flop network does not only underlie olfactory steering, but that it can be a multimodal steering network. If this is correct across insects, we would expect flip-flop neurons to switch state in response to any stimulus that elicits targeted locomotion (excluding taxis behaviors). Here, we discuss three examples of targeted locomotion: straight line orientation, migration, and path integration.

Dung beetles perform short-distance, straight-line orientation when rolling their ball away from the dung pile (Dacke, Nilsson et al., 2003). To keep their path straight, they rely on skylight cues such as the position of the sun, the polarization pattern of the sky, and the sky's spectral gradient (Dacke, Nordström et al., 2003; Dacke, Byrne, Scholtz, & Warrant, 2004; Dacke, Baird, Byrne, Scholtz, & Warrant, 2013; el Jundi et al., 2014). These cues are integrated in the CX to generate a current heading (el Jundi et al., 2016), which can be used to steer the animal along its straight path and adjust for deviations. We therefore expect that dung beetle flip-flop neurons should respond to a change of the skylight cue, such as a sudden rotation of the polarization pattern, with a state change.

When it comes to long-distance migration, the monarch butterfly and the bogong moth are well-known insect models for diurnal and nocturnal migration, respectively (Reppert, Guerra, & Merlin, 2016; Warrant et al., 2016). The monarch butterfly uses a time-compensated sun compass, as well as the geomagnetic field to migrate from its breeding grounds in North America to overwintering regions in central Mexico (Mouritsen & Frost, 2002; Guerra, Gegear, & Reppert, 2014). The bogong moth uses the geomagnetic field in combination with visual landmarks to migrate from its breeding grounds in southern Queensland and western New South Wales (Australia) to its overwintering sites in the Australian Alps (Dreyer et al., 2018). The CX path integration network processes sun compass information and has been proposed to be a possible substrate for computing long-distance migration (Heinze & Reppert, 2011; Honkanen et al., 2019), thus making it likely that the resulting steering commands are passed on to LAL descending neurons. We would expect flip-flop neurons in the monarch butterfly and the bogong moth to switch state in response to sudden changes in the skylight cues or landmark configuration that they use to orient. Furthermore, many flying insects use optic flow for flight control, including moths (Fry, Rohrseitz, Straw, & Dickinson, 2009; Weir, Schnell, & Dickinson, 2014; Stöckl & Kelber, 2019; Mauss & Borst, 2020). One might therefore expect flip-flop neurons to also respond to a change in the rotational component of optic flow with a state change.

Finally, path-integrating ants and bees are obvious targets for measuring flip-flop neuron responses, considering that we use the PI network as an input for our steering models. However, since the flip-flop neurons are not driven directly by sensory input that can be controlled in an experimental situation, but rather by a memory state, it is more difficult to test how flip-flop neurons respond during homing. One could, however, test optic flow cues and compass cues separately to dissect how the different components of the path integrator drive the flip-flop neurons. Alternatively, it may be possible to perform extracellular tetrode recordings from flip-flop neurons during natural homing on a trackball (Dahmen, Wahl, Pfeffer, Mallot, & Wittlinger, 2017).

Of course, motor control mediated by descending neurons is much more complex than the simple model presented here. In the silkworm moth, several other neuron types were described to play a role in pheromone-mediated steering, which can be added to the model for increased complexity and biological relevance. Additionally, complex motor patterns are most likely mediated by not just one cell type but by a population code across a number of descending neurons (Namiki, Dickinson et al., 2018). Creating larger and more integrated models of this form is a useful tool for a fuller understanding of these complex interactions.

We believe that the model we have developed and presented here is one small step toward understanding the connection between the heading direction system in the CX and downstream motor centers. Importantly, the approach we have taken to develop this model is flexible and suitable for a wide range of model features. To date, this is the most complex insect-based model developed using the Nengo neural modeling software. However, Nengo has also been used for a wide variety of mammal-based models, including Spaun, a large-scale model of the human brain (Eliasmith et al., 2012). While modeling insect brains offers different challenges than mammalian brains, we believe our work has shown that this sort of large-scale model is possible and can lead to more realistic behavior than some traditional modeling approaches.

Developing a more complete model cannot be done by a single group of researchers. We have made our model freely available at https://github.com/stanleyheinze/insect_steering, and we hope that a community of researchers can, over time, add neuron types and neural systems to increase the complexity of the model and advance our understanding of this general steering system in insects.

We thank the organizers and participants of the 2016 Nengo summer school, in particular Ben Morcos and Xuan Choo, for contributing to the initial moth model and for helpful discussions. We are also grateful to Noriyasu Ando for sharing his data with us.