We revisit the perceptual crossing simulation studies, which are aimed at challenging methodological individualism in the analysis of social cognition by studying multi-agent real-time interactions. To date, all of these simulation studies have reported that it is practically impossible to evolve artificially a robust behavioral strategy without introducing temporal delays into the simulation. Also, all of the studies report on a single strategy: a perpetually crossing agent pair. Here, we systematically report on the evolutionary success of neural circuits on the perceptual crossing task, with and without sensory delay. We also report on two different strategies in the ensemble of successful solutions, only one of which had been discussed in the literature previously.

What insights can statistical analysis of the time series recordings of neurons and brain regions during behavior give about the neural basis of behavior? With the increasing amount of whole-brain imaging data becoming available, the importance of addressing this unanswered theoretical challenge has become increasingly urgent. We propose a computational neuroethology approach to begin to address this challenge. We evolve dynamical recurrent neural networks to be capable of performing multiple tasks. We then analyze the neural activity using popular network neuroscience tools, specifically functional connectivity using Pearson’s correlation, mutual information, and transfer entropy. We compare the results from these tools against a series of informational lesions, as a way to reveal their degree of approximation to the ground-truth. Our initial analysis reveals an overwhelming large gap between the insights gained from statistical inference of the functionality of the circuits based on neural activity and the actual functionality of the circuits as revealed by mechanistic interventions.

In order to make lifelike, versatile learning adaptive in the artificial domain, one needs a very diverse set of behaviors to learn. We propose a parameterized distribution of classic control-style tasks with minimal information shared between tasks. We discuss what makes a task trivial and offer a basic metric, time in convergence, that measures triviality. We then investigate analytic and empirical approaches to generating reward structures for tasks based on their dynamics in order to minimize triviality. Contrary to our expectations, populations evolved on reward structures that incentivized the most stable locations in state space spend the least time in convergence as we have defined it, because of the outsized importance our metric assigns to behavior fine-tuning in these contexts. This work paves the way towards an understanding of which task distributions enable the development of learning.

Artificial Life has a long tradition of studying the interaction between learning and evolution. And, thanks to the increase in the use of individual learning techniques in Artificial Intelligence, there has been a recent revival of work combining individual and evolutionary learning. Despite the breadth of work in this area, the exact trade-offs between these two forms of learning remain unclear. In this work, we systematically examine the effect of task difficulty, the individual learning approach, and the form of inheritance on the performance of the population across different combinations of learning and evolution. We analyze in depth the conditions in which hybrid strategies that combine lifetime and evolutionary learning outperform either lifetime or evolutionary learning in isolation. We also discuss the importance of these results in both a biological and algorithmic context.

Many lifeforms are found in patches that other lifeforms forage for and consume. Here we explore how the patchiness of the former and cognition of the latter may emerge through mutual interaction in an agent-based model. We use a simple 2D grid world consisting of two types of agents—plants (prey) and animals (predators). Across three experiments, we investigate how cognition of animals influences patchiness of plants and evolves in response to it. Here, cognition is a probabilistic model with two parameters, one for distance of perception and the other for determinacy versus stochasticity of movement. We found that plant patchiness emerged alongside the evolution of animal cognition. In addition, greater distance of perception reduced patchiness, while greater determinacy of movement increased patchiness. Conversely, greater patchiness of plants led animals to evolve perception across greater distances but also led to evolution of less deterministic foraging. Environmental patchiness and foraging cognition thus appeared to mutually create a stable dynamic interaction leading to a self-regulating system.

Understanding how brains and environments give rise to behavior is a subject of great multidisciplinary interest. C. elegans is well-suited for this work because of its relatively rich behavioral repertoire and tractable connectome. The chemotaxis of C. elegans is comprised of two complimentary strategies - “weathervane” (klinotaxis) and “pirouette” (klinokinesis) - that operate in parallel with one another. The present work seeks to characterize each strategy and its contribution to the overall chemotaxis behavior. We find that the contribution of klinotaxis is the primary contributor of chemotaxis performance in most environments, but that klinokinesis is effective in environments with faint stimuli, have few gradient sources or are noisy, particularly when it is integrating sensed concentration over a longer time-scale.

Living organisms learn on multiple time scales: evolutionary as well as individual-lifetime learning. These two learning modes are complementary: the innate phenotypes developed through evolution significantly influence lifetime learning. However, it is still unclear how these two learning methods interact and whether there is a benefit to part of the system being optimized on a different time scale using a population-based approach while the rest of it is trained on a different time-scale using an individualistic learning algorithm. In this work, we study the benefits of such a hybrid approach using an actor-critic framework where the critic part of an agent is optimized over evolutionary time based on its ability to train the actor part of an agent during its lifetime. Typically, critics are optimized on the same time-scale as the actor using the Bellman equation to represent long-term expected reward. We show that evolution can find a variety of different solutions that can still enable an actor to learn to perform a behavior during its lifetime. We also show that although the solutions found by evolution represent different functions, they all provide similar training signals during the lifetime. This suggests that learning on multiple time-scales can effectively simplify the overall optimization process in the actor-critic framework by finding one of many solutions that can still train an actor just as well. Furthermore, analysis of the evolved critics can yield additional possibilities for reinforcement learning beyond the Bellman equation.

Living organisms perform multiple tasks, often using the same or shared neural networks. Such multifunctional neural networks are composed of neurons that contribute to different degrees in the different behaviors. In this work, we take a computational modeling approach to evaluate the extent to which neural resources are specialized or shared across different behaviors. To this end, we develop multifunctional feed-forward neural networks that are capable of performing three control tasks: inverted pendulum, cartpole balancing and single-legged walker. We then perform information lesions of individual neurons to determine their contribution to each task. Following that, we investigate the ability of two commonly used methods to estimate a neuron's contribution from its activity: neural variability and mutual information. Our study reveals the following: First, the same feed-forward neural network is capable of reusing its hidden layer neurons to perform multiple behaviors; second, information lesions reveal that the same behaviors are performed with different levels of reuse in different neural networks; and finally, mutual information is a better estimator of a neuron's contribution to a task than neural variability.

Neuromodulation is a pervasive biological process impacting neural activity at many scales. Changes in the concentration of a single neuromodulator can drastically alter the dynamics of a circuit. Nevertheless, how circuits can be both sensitive to the effects of neuromodulators, yet maintain stable behaviors in the face of constantly changing concentrations of them, is still poorly understood. Past work addressing this has focused on isolated circuits or individual neurons. In this paper, we study the effects of neuromodulation in the context of a complete brain-body-environment model. We use a genetic algorithm to find configurations of a dynamical neural network able to walk with and without the presence of an extrinsic neuromodulatory signal. We analyze, in some detail, networks, which break and cope under the effects of neuromodulation. We identify common stability mechanisms among successful networks, which correspond to previously proposed ideas. In addition, results indicate that proprioceptive feedback provides a stability mechanism for coping with neuromodulation that has not previously been considered in the literature.

A popular hypothesis suggests that the nervous system of different organisms, from neural tissue to whole brains, may operate at or near a critical point. During the last decade, maximum entropy techniques have allowed to go beyond merely finding statistical signatures of criticality, to models directly inferred from data recorded in neural cultures, providing stronger evidence of criticality in neural activity. Nevertheless, these modeling techniques are restricted to neural cultures and have not been extended to neural tissue in living organisms. In this paper, we extend this line of research by analyzing signatures of criticality in a pairwise maximum entropy model inferred from neural recordings of C. elegans during freely-moving locomotion. From the analysis of the inferred models we find some signatures of criticality, as a divergence of the heat capacity of the system. Other indicators, such as Zipf’s distributions, were not found. However, inspecting a similar analysis based in a 2D lattice Ising model we suggest that this could be due to the restricted number of samples in our data set. The availability of larger recordings of the C. elegans neural system during free locomotion could provide more conclusive results.

With 302 neurons and a fully reconstructed connectome, Caernohabditis elegans is an ideal candidate organism to study how behavior is grounded in the interaction between an organism's brain, its body, and its environment. Since nearly its entire behavioral repertoire is expressed through movement, understanding the neuromechanical basis of locomotion is especially critical as a foundation upon which analyses of all other behaviors must build. In this extended abstract, we report on the evolution and analysis of an integrated neuromechanical model of forward locomotion.