An understanding of analogy and the multiple realizability of concepts, ideas, and experience is necessary to understand cognition and the generation of behavior even at the most abstract levels. One of the most fundamental questions one can ask about a pair of neural circuits is whether they are doing the same thing or different things. Our work addresses this question by applying a model of sequential narrative analogy, Net-MATCH, to neural circuits evolved to perform a simple locomotion task. Along the way, we develop a measure of the “experience” of a neural circuit performing a behavior we call its functional trace. We find (i) that Net-MATCH reports strong analogies between some, but not all, neural circuits that perform the walking behavior, (ii) that it finds stronger analogies between circuits of the same class (as reported in previous work on this problem space) than circuits of different classes, and (iii) that it reveals strong analogies between circuits of the previously-reported BS-switch and SW-switch classes, even though these classes are of different circuit sizes. We conclude that Net-MATCH is a powerful tool for understanding the multiple realizability of behavior.

In order to make lifelike, versatile learning adaptive in the artificial domain, one needs a very diverse set of behaviors to learn. We propose a parameterized distribution of classic control-style tasks with minimal information shared between tasks. We discuss what makes a task trivial and offer a basic metric, time in convergence, that measures triviality. We then investigate analytic and empirical approaches to generating reward structures for tasks based on their dynamics in order to minimize triviality. Contrary to our expectations, populations evolved on reward structures that incentivized the most stable locations in state space spend the least time in convergence as we have defined it, because of the outsized importance our metric assigns to behavior fine-tuning in these contexts. This work paves the way towards an understanding of which task distributions enable the development of learning.

Living organisms learn on multiple time scales: evolutionary as well as individual-lifetime learning. These two learning modes are complementary: the innate phenotypes developed through evolution significantly influence lifetime learning. However, it is still unclear how these two learning methods interact and whether there is a benefit to part of the system being optimized on a different time scale using a population-based approach while the rest of it is trained on a different time-scale using an individualistic learning algorithm. In this work, we study the benefits of such a hybrid approach using an actor-critic framework where the critic part of an agent is optimized over evolutionary time based on its ability to train the actor part of an agent during its lifetime. Typically, critics are optimized on the same time-scale as the actor using the Bellman equation to represent long-term expected reward. We show that evolution can find a variety of different solutions that can still enable an actor to learn to perform a behavior during its lifetime. We also show that although the solutions found by evolution represent different functions, they all provide similar training signals during the lifetime. This suggests that learning on multiple time-scales can effectively simplify the overall optimization process in the actor-critic framework by finding one of many solutions that can still train an actor just as well. Furthermore, analysis of the evolved critics can yield additional possibilities for reinforcement learning beyond the Bellman equation.