We consider the dynamics of artificial chemistry systems consisting of small, interacting neural-network particles. Although recent explorations into properties of such systems have shown interesting phenomena, like self-replication tendencies, social interplay, and the ability for multi-objective applications, most of these settings are reasoned about in the abstract weight space. We extend this setup to involve an applied, stateful positioning task with mutual dependencies and show that stable configurations can be found jointly in both the weight space and 3D space. We show that the main contributing factor is enabling the networks to self-adapt their interaction rates depending on their internal stability or their ability to position themselves correctly. We find that this method effectively prepares the network assembly against potentially destabilizing interactions, promoting emergent stability while preventing convergence to trivial states.

A recent branch of research in artificial life has constructed artificial chemistry systems whose particles are dynamic neural networks. These particles can be applied to each other and show a tendency towards self-replication of their weight values. We define new interactions for said particles that allow them to recognize one another and learn predictors for each other’s behavior. For instance, each particle minimizes its surprise when observing another particle’s behavior. Given a special catalyst particle to exert evolutionary selection pressure on the soup of particles, these ‘social’ interactions are sufficient to produce emergent behavior similar to the stability pattern previously only achieved via explicit self-replication training.

Self-replicating neural networks can be trained to output a representation of themselves, making them navigate towards non-trivial fixpoints in their weight space. We explore the problem of adding a secondary functionality to the primary task of replication. We find a successful solution in training the networks with separate input/output vectors for one network trained in both tasks so that the additional task does not hinder (and even stabilizes) the self-replication task. Furthermore, we observe the interaction of our goal-networks in an artificial chemistry environment. We examine the influence of different action parameters on the population and their effects on the group's learning capability. Lastly we show the possibility of safely guiding the whole group to goal-fulfilling weight configurations via the inclusion of one specially-developed guiding particle that is able to propagate a secondary task to its peers.

A key challenge in AI is the development of algorithms that are capable of cooperative behavior in interactions involving multiple independent machines or individuals. Of particular interest are social dilemmas, which are situations that raise tension between an individual's best choice and the desirable outcome in terms of the group. Although such scenarios have been studied increasingly within the AI community recently, there are still many open questions on which aspects drive cooperative behavior in a particular situation. Based on the insights from behavioral experiments that have suggested positive effects of penalty mechanisms towards cooperation, in this work we adopt the notion of penalties by enabling independent and adaptive agents to penalize others. To that end, we extend agents’ action spaces with penalty actions and define a negative real-valued punishment value. We utilize reinforcement learning to simulate a process of repeated interaction between independent agents, learning by means of trial-and-error. Our evaluation considers different two player social dilemmas, and the N-player Prisoner's Dilemma with up to 128 independent agents, where we demonstrate that the proposed mechanism combined with decentralized learning significantly increases cooperation within all experiments.

We implement a probabilistic version of the game of life on a quantum annealer in a very direct manner, demonstrating possible use cases for (currently very noisy) quantum annealing devices in the simulation of (usually very stochastic) artificial life and complex systems in general.

This paper considers sustainable and cooperative behavior in multi-agent systems. In the proposed predator-prey simulation, multiple selfish predators can learn to act sustainably by maintaining a herd of reproducing prey and further hunt cooperatively for long term benefit. Since the predators face starvation pressure, the scenario can also turn in a tragedy of the commons if selfish individuals decide to greedily hunt down the prey population before their conspecifics do, ultimately leading to extinction of prey and predators. This paper uses Multi-Agent Reinforcement Learning to overcome a collapse of the simulated ecosystem, analyzes the impact factors over multiple dimensions and proposes suitable metrics. We show that up to three predators are able to learn sustainable behavior in form of collective herding under starvation pressure. Complex cooperation in form of group hunting emerges between the predators as their speed is handicapped and the prey is given more degrees of freedom to escape. The implementation of environment and reinforcement learning pipeline is available online.

This paper applies reinforcement learning to train a predator to hunt multiple prey, which are able to reproduce, in a 2D simulation. It is shown that, using methods of curriculum learning, long-term reward discounting and stacked observations, a reinforcement-learning-based predator can achieve an economic strategy: Only hunt when there is still prey left to reproduce in order to maintain the population. Hence, purely selfish goals are sufficient to motivate a reinforcement learning agent for long-term planning and keeping a certain balance with its environment by not depleting its resources. While a comparably simple reinforcement learning algorithm achieves such behavior in the present scenario, providing a suitable amount of past and predictive information turns out to be crucial for the training success.

Flocking or swarm behavior is a widely observed phenomenon in nature. Although the entities might have self-interested goals like evading predators or foraging, they group themselves together because a collaborative observation is superior to the observation of a single individual. In this paper, we evaluate the emergence of swarms in a foraging task using multi-agent reinforcement learning (MARL). Every individual can move freely in a continuous space with the objective to follow a moving target object in a partially observable environment. The individuals are self-interested as there is no explicit incentive to collaborate with each other. However, our evaluation shows that these individuals learn to form swarms out of self-interest and learn to orient themselves to each other in order to find the target object even when it is out of sight for most individuals.

In nature, flocking or swarm behavior is observed in many species as it has beneficial properties like reducing the probability of being caught by a predator. In this paper, we propose SELFish (Swarm Emergent Learning Fish), an approach with multiple autonomous agents which can freely move in a continuous space with the objective to avoid being caught by a present predator. The predator has the property that it might get distracted by multiple possible preys in its vicinity. We show that this property in interaction with self-interested agents which are trained with reinforcement learning solely to survive as long as possible leads to flocking behavior similar to Boids, a common simulation for flocking behavior. Furthermore we present interesting insights into the swarming behavior and into the process of agents being caught in our modeled environment.

The foundation of biological structures is self-replication. Neural networks are the prime structure used for the emergent construction of complex behavior in computers. We analyze how various network types lend themselves to self-replication. We argue that backpropagation is the natural way to navigate the space of network weights and show how it allows non-trivial self-replicators to arise naturally. We then extend the setting to construct an artificial chemistry environment of several neural networks.