## 1 Introduction

The idea that agents may be socially attracted to each other by way of actively seeking each other out in order to benefit from the proximity of others—be it to avoid predators, breed, or cooperatively raise their young, or to discover new resources or habitats—is a well-established one [1, 2]. However, it is difficult to establish precisely why and when social information leads to increased social interaction and pro-social behavior, social information here being defined as information derived from the behaviors, actions, cues, or signals of other agents [23]. As social information necessarily involves the direct or indirect broadcasting of information into the public domain, it is sometimes known as (or conflated with) public information [5]. Here we will use term social information when describing any information about an individual that is broadcast into the public domain.

### 1.1 General Hypotheses for the Emergence of Social Interaction Promoting Behavior

In reviewing social information use, Valone [36] outlines three general hypotheses to explain why individuals might prefer to settle near conspecifics (leading to what may be described as habitat copying via local enhancement):

• 1.

Individual fitness is enhanced via the Allee effect [1, 2, 32], which is defined by Stephens et al. [34], pg. 186 as “a positive relationship between any component of individual fitness and either numbers or density of conspecifics.” Allee observed that individuals were better able to survive and reproduce when found in groups, and concluded that there is a positive correlation between population density or group size and individual fitness (known as the Allee effect). If this effect holds true, we would expect there to be selection pressure in favor of agents being in close proximity to one another; increased use of social information may therefore be a result of increased social interaction due to agent proximity.

• 2.

Social information based resource discovery results in a reduction in search costs, enabling a more efficient use of energy [17, 32]. As social information may be used to reduce search costs and to increase the chance of experiencing new resources that might otherwise have been overlooked, increased agent interaction may result from a selective pressure to obtain social information, rather than increased social information use being a secondary consequence of increased agent interaction itself; the Allee effect would then result as a consequence of this selective pressure to access social information.

• 3.

Individuals use the presence of other (established) individuals as an indicator of the high quality of a habitat without necessarily having to rely on their own (possibly incomplete or poor) evaluation of the habitat [35, 39]. Here social information not only reduces the search costs when discovering resources, but also enables individuals to derive the quality of an unfamiliar resource from social information about the action, state, or presence of others. Again, agent interaction and the Allee effect result as consequences of selective pressures in favor of social information use, rather than social information use resulting as a consequence of a selective pressure in favor of agent interaction. This hypothesis is similar to hypothesis 2 (listed above), but differs subtly; hypothesis 2 is associated with simply discovering resources, whereas this hypothesis is associated with judging the quality of a resource once found. This hypothesis may be a direct result of hypothesis 2.

### 1.2 Behavior in the Presence of Social Information

Here we assess three questions regarding agent behavior in the presence of social information. Firstly, we assess whether the well-established notion that social information leads to behaviors that promote agent interaction is true in simple artificial evolutionary systems such as the one used by Borg and Channon [6]. Secondly, we assess whether agents' private-information reliability (or environmental predictability) affects agent interaction and social information use. Finally, we assess whether any observed social behaviors (i.e., behaviors resulting from the use of social information) can be seen to persist even when social information use does not lead to improved task performance—that is to say, when agents with access to social information no longer perform better than agents with no access to social information on a simple food-foraging task, where performance is measured by the proportion of eating activity dedicated to consuming positive foods compared to negative foods.

The question of the persistence of what may be described as non-adaptive social information use, or social learning, was addressed by Higgs [20] in his meme-based simulation study of learning by imitation. One of the many things Higgs concluded was that memes (discrete, replicating, units of culture [3, 4, 12, 13]), even when they provided negative biological fitness, still led to the evolution of imitation. In Higgs' model individuals had both a biological and a cultural fitness. Both of these fitness values were determined by the set of memes held by an individual, with reproduction being determined by biological fitness, and the chance of being imitated being determined by cultural fitness. In one of Higgs' test cases the biological fitness provided by a meme was the reverse of the cultural fitness, resulting in all biologically fit memes being culturally unfit and all culturally fit memes being biologically unfit—even in this test case, imitative learning evolved. This suggests that behavior that increases social interactions may still be adaptive even when task performance is poor.

Higgs' [20] result is not necessarily surprising, as it is more than reasonable to expect to see agents with access to social information of any kind seeking this information out regardless of the contribution it makes to fitness, provided some of the social information could provide an adaptive advantage. Bullinaria [10], pg. 382 rationalizes this expectation by stating: “If there exists a set of memes with a range of positive and negative contributions to the overall performance, then not imitating them will leave performance at some baseline, while imitating them will result in a range of performance levels above and below that baseline. Any selection on the basis of performance will then favor those individuals that have imitated the good memes, and hence favor higher imitation rates.” Therefore, we can see why agents may wish to collect around sources of information; sometimes that information will be useful, so gaining access to it is important. We would therefore expect to see agents attempting to find sources of information even when obtaining that information does not necessarily lead to an improved performance. Agent and social interaction for the purpose of habitat copying is also found to be adaptive in highly variable environments [38], though with the potential pitfall of population collapse during overly conformist social interaction [8, 40]. It has also been noted by Rendell et al. [28] that strategies that rely heavily on social learning seem to be remarkably successful, even when information obtained from non-social sources is no more costly than social information. We would therefore expect behaviors that maximize access to social information to emerge.

In the model setup developed by Borg and Channon [6], which forms the basis for this work, there are a large number of possible food resources available to agents, resulting in agents often being uncertain about whether any given food resource will provide a positive or negative amount of energy. As environments in that model become more difficult, a strategy whereby all food is ignored may evolve, but this strategy would always be outperformed by a strategy that sought to minimize uncertainty about available food resources in order to discover positive-energy-providing resources. Social information, especially about the performance or fitness of an agent, may therefore be sought in order to allow for decisions on whether to consume any given food resource to be influenced by others, thus reducing uncertainty about the safety of a new food resource. This kind of social information seeking behavior, directed towards information about new or novel food resources, in often seen in Norway rats [15, 27], though it is interesting to note that this social behavior is only used to develop food preferences and not food aversions; this property of rat social behavior has been suggested to result from the high lethality associated with poor food choices in rat populations [27], thus resulting in very little social information about negative food resources being available to the population. We may see a similar scenario in the more difficult environments presented here, providing a continued pressure for social behavior under extreme environmental difficulty. Van Bergen et al. [37] report that when individually learned information is less reliable, nine-spined stickleback fish tend to use social rather than individually learned information; this could be rephrased as stating that social learning is more likely to take place when a task is difficult to learn individually. Therefore, it is not unreasonable here to expect agents in populations that have access to social information to seek this information out in order to reduce the unreliability of their own internal models of the world; it is far easier to evolve prestige-based social strategies such as “trust older individuals” or “trust successful individuals” [19, 25], or conformist social strategies such as “trust the majority” [18, 25], than to evolve a rule about each possible food resource or situation one might experience, especially when it is likely that any given food resource or situation is new to an agent and therefore is yet to be evaluated.

### 1.3 Previous Work: The EnVar Model

The work discussed here follows on from previous work by Borg and Channon [6]. In that work an artificial life model, called EnVar, was created to investigate the evolutionary adaptation to social information use without learning. The question posed by Borg and Channon was: Does the addition of social information enable agents to evolve to perform better on a simple food foraging task than when social information is not available?

The EnVar model places a population of agents in a 2D simulated environment containing a large variety of food (plant) resources. Food resources are recognized by agents by their color (RGB values), with foods grouped into species of plants based on their color. Some of the plant species provided positive energy when consumed, and others provided negative energy. The simplest task tested involved two food species, with a 1 : 1 ratio of positive to negative species; the most difficult task involved ten food species with a 1 : 9 ratio of positive to negative food species. A series of different populations with access to different types of social information were tested, with performance on the task being measured by how much time agents spent consuming positive food resources compared to how much time they spent consuming negative food resources. All agents had a limited amount of energy, which was lost through eating negative food and regained through eating positive food. Residual amounts of energy were also lost when agents simply did nothing or when they were moving, with energy lost due to movement being greater than energy lost due to waiting. Agents were replaced when they ran out of energy, the replacement agents being the progeny of two surviving agents from the population.

Each population of agents was set up to use one of five social information strategies. One of the these strategies involved no social information at all, whereas the other four involved social information about either the activity or the state of other agents. The two social information strategies associated with activity were presence, where an agent could only see whether another agent was present or not, and action, where an agent could see whether another agent was eating, waiting, or moving (the only three actions available to agents in the model). The two social information strategies associated with the agent state were health, where an agent could see the battery level of another agent, and age, where an agent could see how old others agents were.

As expected, absolute performance on the task dropped with increasing environmental difficulty in all cases. However, populations using social information did outperform non-social populations in simpler environments, thus demonstrating an evolutionary advantage to using social information. In some cases, social information also enabled populations to maintain positive task performance across a wider range of environments; the best social information strategy observed was able to perform well (eating more positive food than negative food) up to an environment with five negative food species to one positive food species; non-social populations were only able to achieve a positive task performance up to an environment with an 1: 3 ratio of positive to negative food species. Despite populations making use of social information generally outperforming non-social populations, there was no significant difference between social and non-social populations in the more difficult environments that were tested.

The model used in this work, including the social information strategies implemented, exactly matches Borg and Channon's [6] model (more details on this model can be found later in this article). Therefore, all hypotheses should be considered in the context of that model and results.

### 1.4 Hypotheses

The large amount of evidence suggesting the persistence of social information promoting behaviors in unreliable and challenging environments, and evidence from simulations that social learning mechanisms such as imitation provide a selective advantage even when the information being obtained is not necessarily fitness increasing, along with the well-established principle that the desire to obtain social information leads to agent interaction, lead us to postulate the following hypotheses to be assessed here.

• 1.

Social information should lead to behaviors that result in increased agent interaction (i.e., movement to seek social interactions): We will test this hypothesis by comparing the amounts of movement undertaken by agents with and without social information. If we see a significant difference in the amount of movement, we will then assess how much time agents from social populations spend around other agents. We require a significantly larger number of movement actions combined with agent interaction to demonstrate not only socially influenced interaction, but also behaviors that promote social interactions. Sergio and Newton [31] provide evidence that in some cases even simple information such as the presence of other individuals (the occupancy) can be a suitable indicator of resource quality and therefore enough to lead to agents coalescing around a food source. Therefore we would expect this hypothesis to hold true for all social information strategies presented here, though when the presence of another agent is used as a source of social information, some measure of resource quality may still be required, as no information about the success or state of the agent present at the resource is available to act as a proxy for resource quality [36].

• 2.

Social interaction between agents will be more likely when environments are more unpredictable, and less likely when environments are more predictable. In the model environment used here it could be argued that the more difficult environments are more predictable. The most difficult environment tested here has a ratio of one positive food resource to every nine negative food resources; therefore agents have a 90% chance of correctly guessing that a food resource will be dangerous. We may therefore expect agent social interaction (should it be seen) to be at its highest in lower-difficulty environments, despite the possibility of non-social agents also performing well in these environments. From an artificial life and evolutionary robotics perspective it would be useful to know under which conditions pro-social behaviors, such as agent social interaction and cooperative foraging, may emerge.

• 3.

Behaviors resulting in increased agent interaction will persist (though at reduced levels) even when task performance is poor, poor task performance being characterized by agents spending more time eating negative food than eating positive food. The adaptive value of social information, even when potentially unreliable, should still be high enough to motivate agents to seek others out more often than if social information were not available. In the more difficult environments tested here we would expect social information to be relatively poor, due to the large quantities of negative food resources populating the environment. However, it would still be beneficial for agents to engage in movement sometimes in order to provide potential access to any positive behaviors that might emerge in the population. Therefore we would expect behaviors that encourage social interaction (i.e., movement) to still appear more often in social populations than in non-social ones, in all environments.

We will also go on to assess whether social information leads to any significant difference in the application of the other behaviors available to agents here from that in non-social populations, and whether task performance has any implications for the application of behavior—we are especially interested in assessing whether a change in task performance from the predominantly successful application of eat actions to the predominately unsuccessful application of eat actions is accompanied by any notable transitions in behavior.

## 2 Simulation Model and Experimentation

The experimental setup matches that used by Borg and Channon [6]. Summary tables of the key parameters used can be found in the  Appendix.

Populations of neuroevolutionary agents (making use of the hybrid neural network model known as the shunting model [7, 22, 30, 33, 42, 43]), each population employing a different social information strategy, are tasked with surviving in environments of differing difficulties. In order to test our hypotheses, we test populations of social and non-social agents in a set of increasingly difficult environments, forty populations of each social information strategy being evaluated per environment. Environmental difficulty is dictated by the ratio of positive food resources to negative food resources. The simplest world used here has an equal (1 : 1) ratio of positive food species to negative food species. Tests are made progressively harder by increasing the number of negative food species, while maintaining only one positive food species, resulting in the most difficult world used here having a 1 : 9 ratio of positive to negative food species. All data presented here relates to the final 25 epochs of evolution (of a total of 100 epochs), where population behavior and fitness had broadly stabilized (according to the results of [6]). An epoch here is defined as 1,000 time steps, with a time step being defined as one full simulation loop.

The task world used here is known as EnVar. EnVar is a bounded (non-toroidal) 2D environment containing a variety of consumable resources known as plants. A plants is recognized by agents simply as an RGB value. Plants are divided into a number of species, each with a randomly selected base RGB value. Plants are generated within these RGB regions and identified as belonging to the nearest species according to Euclidean distance in RGB space to a species base. The number of plant species is determined by the test being conducted. In the tests conducted here, the number of species ranges from two to ten. Each plant species is assigned an energy value, which is transferred to agents if the plant of that species is consumed; energy values may be positive or negative.

Notionally the EnVar world is broken up into cells, though here each cell represents a pixel. Plants in the world take up a number of cells, set here to 100, forming a 10 × 10 block, with each block only being able to be eaten a certain number of times before being exhausted (here set to be 200 eating events). Once a plant block has been exhausted, it is no longer consumable and therefore removed from the world to be replaced by a new block from a random plant species somewhere else in the world; this maintains a constant number of food blocks in the world at any time. Agents are permitted to share space with a plant resource, but cannot overlap with each other. This avoids the possibility of agents piling up on top of one another on valuable food resources, which could result in an agent's path to a food resource being blocked by agents already on that resource, though agents cannot intentionally choose to block other agents.

For all tests here, negative food species come with an energy value Eneg = −10.0, and positive food species contribute an energy value of Epos = 1.0 when consumed. This provides a strong evolutionary pressure to avoid eating negative food species. In this work EnVar is set up to create a 700 × 700 pixel cell world, containing five hundred 10 × 10 pixel blocks of plants.

### 2.1 Neuroevolutionary Model

Agents in the EnVar simulation world are grounded 2D simulated agents, controlled by a hybrid neural network architecture known as the shunting model [42, 43]. The shunting model uses two interacting networks to determine agent behaviors, here represented as a discrete set of agent actions. The two interacting networks are known as the decision network and the shunting network. The decision network is simply a feedforward neural network composed of an input layer, one hidden layer, and an output layer. Outputs from the decision network (known as Iota values) are used to produce a locally connected, topologically organized network of neurons known as the shunting network, which simply places and organizes agent preferences for environmental features and states in such a way as to allow the agent to hill-climb in a shunting space (known as the activity landscape) that directly maps onto its immediate neighborhood. The shunting network's weights are fixed for all agents, whereas the decision network is genetically encoded and is subject to change via evolution.

#### 2.1.1 The Shunting Network

The shunting network is a locally connected, topologically-organized network of neurons that was originally used for collision-free motion planning in robots [42, 43] and has subsequently been applied in a number of 2D and 3D artificial life models [6, 7, 22, 30, 33]. Here the shunting network's topology is simply superimposed on the environment, with each cell in the network topology directly relating to a pixel within an agent's visual field. Using a simplified and stable version of the shunting equation developed by Stanton and Channon [33],
$xinew=maxminImin18∑jϵNixj++IimaxI,$
(1)
values for each cell (which can be interpreted as representing an environmental feature or state, and are initially set by the Iota output I obtained from the decision network) are propagated across the cells of the network, producing an activity landscape with peaks and valleys representing desirable and undesirable features in the environment. The result is a landscape that allows the agent to follow a route determined by the higher Iota values while avoiding undesirable valleys. A mockup example of an activity landscape with a snapshot of the visual field it represents can be seen in Figure 1.
Figure 1.

Mockup transition from agent visual field to shunting network activity landscape. The left-hand grid shows the agent's visual field with two plant objects and one other agent occupying the same space as a plant. The right-hand grid shows an example activity landscape for the visual field. The agent determines that an agent on a plant is an interesting feature and therefore assigns it a strong positive Iota value (I), whereas the purple plant is seen negatively and is therefore assigned a strong negative Iota value. These Iota values propagate over the activity landscape using Equation 1. The central agent then chooses to move within its immediate Moore neighborhood to the cell with highest activity value.

Figure 1.

Mockup transition from agent visual field to shunting network activity landscape. The left-hand grid shows the agent's visual field with two plant objects and one other agent occupying the same space as a plant. The right-hand grid shows an example activity landscape for the visual field. The agent determines that an agent on a plant is an interesting feature and therefore assigns it a strong positive Iota value (I), whereas the purple plant is seen negatively and is therefore assigned a strong negative Iota value. These Iota values propagate over the activity landscape using Equation 1. The central agent then chooses to move within its immediate Moore neighborhood to the cell with highest activity value.

In Equation 1 each node in the shunting network corresponds to one pixel within an agent's visual field; xi is the activation of neuron i; Ni is the receptive field of i; the function [x]+ is max(0, x); and Ii is the external input to neuron i (the Iota value). The maximum Iota value is maxI = 15, with the resulting value for $xinew$ also being capped at a minimum Iota value minI = −15. This stops Iota values growing out of control, while providing a large enough maximum value (and a small enough minimum value) to ensure activity propagation across the network. In order to allow propagation to occur within a time step, the shunting equation must be run a number of times; we take this number of iterations to be equal to the diameter of the visual field.

The shunting model implemented here differs in a number of significant ways from previous artificial life implementations [7, 22, 30, 33]. In these previous implementations agents see their entire environment, have a set number of discrete environmental features and states to set Iota values for, and are in the environment alone to complete a predetermined task. Here agents have a limited view of the world, have the possibility of needing to set an Iota value for a plant of any given RGB value, and exist as a population within the environment (leading to possible input states where an agent can be seen at a particular plant). In order to accommodate these differences the shunting model here is run independently for each pixel in an agent's visual field, which is set here to have a radius of 30 pixels from the center of the agent, with information about that pixel being included as part of the agent's decision network input layer. In this way an Iota value is calculated for each unique environmental state within an agent's visual field. This change does not change the resulting behavior of the shunting model or activity landscape, just the way in which information is passed to the shunting network from the decision network.

#### 2.1.2 The Decision Network, Neuroevolution, and Reproduction

Evolution in the model is applied only to the decision network. Here, the decision network is a feedforward neural network composed of seven input nodes, an additional social input node in social information tests, eight hidden units, and two output nodes, resulting in 112–128 weights. Each network layer is fully connected, with floating-point weights in the range [−1 : 1] being directly encoded from an agent's genotype. A standard sigmoid activation function is used at each hidden and output node, though outputs processed for deriving agent actions are then scaled to be within the range [0 : 1] and the Iota output is scaled linearly to be within the range [minI : maxI]. As the agent is expected to produce an Iota value to feed into the shunting network for each unique environmental feature or state within its visual field, inputs into the decision network must accommodate the internal state of the agent, the state of their current environment, and the state of the environmental feature they are assessing; this leads to there being two sets of input nodes. The first set of input nodes are simply plant RGB inputs—if the agent is viewing empty space, these inputs are set to −1, else they are set to be the normalized RGB of the plant being viewed, with RGB values being normalized to be within the range [0 : 1] by way of linear normalization. Following these inputs are a series of generic inputs, which are dependent on the agent's internal state and the current environmental state. These inputs are the agent's current battery level in the normalized range [0 : 1], a moving average of the agent's battery level over the previous 100 time steps, and also the agent's current external environmental state and a moving-average environmental state, which are both set to be +1 and do not change in the tests presented here (the model is set up to accommodate external environmental change, which is not used here). In social information tests agents have an additional input based on the agent being viewed.

The genotype, which is essentially an array of weights, is subjected to both mutation and crossover should a reproduction event take place. The crossover mechanism used here is single-point crossover, with per locus mutation occurring with probability pmut = 1/L, where L is the length of the genotype. Mutation is achieved by way of Gaussian random noise: a value taken from a normal distribution with μ = 0 and σ = 0.01 is either subtracted or added to the floating-point value at the loci to be mutated. All weight values are bounded in the range [−1 : 1]. Reproduction events take place only in response to a death event. Agents can die if they run out of energy, or if they are in the lowest 10% of agents ranked by energy at the end of an epoch. The first method for removing agents from the population ensures that agents cannot remain in the population with no energy, and the second method ensures that space is made for new agents to be created even if the population as a whole is successful at maintaining above-zero energy levels, thus maintaining a selection pressure for task improvement. Neither method of death is directly related to task ability, as it is possible for a good agent to be unlucky and never, or rarely, experience a positive food resource, whereas less able agents may have the luck to be born near an abundance of food resources or relatively close to the end of an epoch. This method of reproduction maintains a constant population size of 200 agents.

The new agent, or child, created to replace the removed agent is the progeny of two agents, of whom one is selected in a tournament, and the other is selected randomly from the remaining population. The tournament selection mechanism applied here takes two agents from the population, compares their current energy levels, and selects the fitter agent (i.e., the agent with the higher energy level) as a parent. As is true in nature, this isn't a perfect measure of fitness, as it is possible the agent is young and therefore has not yet had time to lose significant amounts of energy, or the agent could have simply been lucky or unlucky with available food sources. In general, however, agents with more effective behaviors will on average find themselves with better energy levels than agents with less effective behaviors, thus driving evolution toward behaviors that are more suited to the task or environment at hand. The second parent is selected randomly to ensure the population doesn't become dominated by the progeny of a small subset of the population, thus maintaining a level of exploration in the genotypic search space. New agents are placed in the world within the visual field of one of their parents, selected at random—this places agents within close proximity of each other without the need for agents to explore, providing a pressure against the evolution of exploratory movement to seek out other agents.

### 2.2 Agent Actions and Action Energy Costs

The agents in the model have a set of simple, discrete actions available to them, through the output layer of their decision networks: wait, eat, or move. The decision network has two outputs: an Iota output to be fed into the shunting network, and an eat or wait output. The agent first considers the input state at its current position; if the agent produces an Iota value above the threshold θa = 0.5, it indicates the agent is happy with its current state and position and therefore does not move. The agent's eat or wait output is then considered; if the output produces a value above the threshold θb = 0.5, the agent attempts to eat whatever may be at its current position. Agents are allowed to try to eat at locations where no plant is present, but no benefit is conferred by this action, and the eat action is considered to be an unsuccessful eating attempt rather than a wait action. If an agent decided to eat at a location containing a plant, the plant's energy is transferred to the agent. This does not necessarily lead to the exhaustion of the plant resource. The Iota output is in the range [−1 : 1]; any values in the range [−θa : θa] are evaluated as neutral and resolve to 0. The Iota output is then scaled to be within the range [minI : maxI] for use in the shunting network, whereas the eat or wait output is limited to the range [0 : 1]. If the eat or wait output is below the expected threshold, the agent simply waits at its current location. Waiting and eating both reduce an agent's energy by 0.1 energy units (though eating may result in a net energy gain), and moving uses up 0.2 energy units per time step. Agents will only move if their Iota output for their current location is below the threshold θa. In this case an activity landscape is created, based on the Iota outputs for all visible environmental features. Agents are born with, and are able to achieve, a maximum energy level of 100 units. As epochs here constitute 1,000 time steps, an agent would be able to survive for a maximum of one epoch, or 1,000 time steps, by remaining inactive. In order to avoid agents moving around in circles, or moving backwards and forwards, in neutral space where there is no activity gradient from the activity landscape, consecutive neutral move actions maintain the same direction of travel with probability pdir = 0.9.

### 2.3 Social Information Strategies

The social information strategies explored here, including the no-social strategy, are discussed below:

No Social: No input node is available to the agent to enable social information to be used by the agent's decision network. Agents proceed with no information about other agents. There is very little evidence in nature for agents being totally ignorant of the presence of other agents—this strategy was simply to be used as a baseline to compare the other social information strategies against.

Presence: The social information input node receives an input of +1 if another agent is present within the visual field. No other information about the agent being viewed is used. This strategy is similar to the “inadvertent information” strategy used by agents in the work by Mitri et al. [26], but the agents explored in the work presented here do not have a choice about whether they express social information or not. In nature, in a number of vertebrates [16], the presence of other agents has been established as a key motivator of where to eat or explore. Social facilitation, defined as the mere presence of a demonstrator affecting an observing agent's behavior [21, 29], is an example of a social learning strategy, observed in nature, arising from the mere presence of other agents.

Action: An input represents the current action state of the agent being viewed. The wait action is input as a value of 0, eat as 0.5, and move as 1. Amalgamating these action inputs into one input rather than two or three categorical inputs, while not ideal, was implemented in order to ensure that the input layer size for all social strategies was the same. Being able to observe and interpret the activity or actions of other agents can lead to a variety of social learning strategies seen in nature—these strategies include observational conditioning, social enhancement, response facilitation, and contextual imitation [21, 29].

Health: The current energy levels of the agent being viewed are normalized to be within the range [0 : 1] and input to the viewing agent's decision network. Health information here is used as a possible proxy for the success of agents, though a noisy one, as high energy levels could indicate that the agent is young (and yet to expend any energy) or lucky, as well as indicating that an agent has evolved a suite of adaptive behaviors that minimizes energy use and maximizes successful eating events. The social learning strategy “copy successful individuals” is seen regularly in nature [25], and is well established in theoretical modeling as a viable social learning strategy [9].

Age: The age (in time steps) of the agent being viewed is normalized using a hyperbolic tangent function of the logarithm of the age, which is then normalized to be within the range [0 : 1] (with 1 being asymptotic):
$inputa=tanhloga+1/2.$
(2)
Normalizing age in this way is necessary because agents may live for the entire duration of the simulation, and are not selected against based upon their age. In equation 2, a represents agent age in time steps. Using information about the age of other agents can result in a “copy older individual” social learning strategy [25]; such strategies are observed in mate choice copying in fish [14, 24]. As avoiding being removed from the population is also an indication of successful behavior, copying individuals can also be seen as another form of the “copy successful individuals” strategy.

It is worth noting that despite references to social learning, this work contains no actual learning; therefore we would not expect complex social strategies such as those seen in nature to emerge here. All references to social learning in nature here are instead supposed to show why the social information being used here may be justified as forming the basis of more complex social learning strategies seen in nature.

## 3 Results

### 3.1 Action Profiles

Figure 2 shows the median action profiles for each social information strategy applied here, an action profile being the proportion of all actions each individual action contributed. The most immediate difference between the social information using populations and the non-social populations, from Figure 2, is the application of the move action. While all populations show a reduction in movement (as environmental difficulty increases), with an accompanying increase in waiting, non-social populations have extremely low levels of movement, even in environments of lower difficulty, compared to social information populations. In social populations movement is applied more frequently than waiting in lower-difficulty environments. This suggests that the increased performance associated with populations that use social information in simpler environments, seen previously [6], is a consequence of this greater willingness to move, either to find new food resources or to find new sources of social information. As the only difference between social and non-social populations is the addition of social inputs to agent neural networks, movement to seek new sources of information is probably closer to the truth. As agents in all populations spend the majority of their time in simpler environments eating, any movement motivated by the desire to be around other agents would lead to a secondary consequence of being around more food resources, enabling agents who are less able to distinguish between positive and negative food resources to defer some of their judgments on the likely payoff of a food resource, and instead rely on the social information being provided by the agents they now find themselves around to make more informed decisions. However, it is not clear from Figure 2 whether or not this difference in movement between non-social and social populations is significant, and whether this additional movement does lead to more opportunities for social information use.

Figure 2.

Median agent action profiles for each social information strategy over each environment difficulty.

Figure 2.

Median agent action profiles for each social information strategy over each environment difficulty.

The immediate difference in movement behavior between non-social and social populations seen in Figure 2 is demonstrated to be significant by way of Mann-Whitney U tests between the resulting application of move actions for social populations and for non-social populations; this can be seen in Figure 3. The continued significance of the difference between social and non-social populations regarding movement is in contrast to the general lack of significance in task performance difference between social and non-social populations in environments past environment 2 (as seen in [6]); these results indicate that the introduction of social information leads to behavioral differences that persist even when these behaviors do not result in improved task performance.

Figure 3.

Median move actions for each social information strategy over all environments. Data points on the primary y axis represent the median proportion of the move action. Data points on the secondary y axis represent the Z-score value from a Mann-Whitney U test comparing, for each environment, the median actions for the two social information strategies presented. Z-scores that indicate statistically significant p-values are highlighted on the x axis.

Figure 3.

Median move actions for each social information strategy over all environments. Data points on the primary y axis represent the median proportion of the move action. Data points on the secondary y axis represent the Z-score value from a Mann-Whitney U test comparing, for each environment, the median actions for the two social information strategies presented. Z-scores that indicate statistically significant p-values are highlighted on the x axis.

Regarding the other actions available to agents—eating (see Figure 4) and waiting (see Figure 5)—neither shows any particular significant differences (where p < 0.01) between social and non-social populations except in environment 1, where waiting actions for all social populations are applied significantly less than in non-social populations (p < 0.01), and eating actions are applied significantly less for social populations using the presence and action strategies than in non-social populations (p < 0.01). This broad lack of significant differences, beyond environment 1, between non-social and social populations for eating and waiting further demonstrates that movement is the primary driving force in the improved task performance seen in earlier environments, especially in environment 2, where only movement is significantly different, despite previous work [6] showing a significant difference in task performance. It should, however, be noted that in environment 1 social information availability also leads to significantly different eating and waiting behaviors, indicating that some adaptive action profile across actions is available to drive improved task performance, rather than just a reliance on movement behavior.

Figure 4.

Median eat actions for each social information strategy over all environments. Data points on the primary y axis represent the median proportion of the eat action. Data points on the secondary y axis represent the Z-score value from a Mann-Whitney U test comparing, for each environment, the median actions for the two social information strategies presented. Z-scores that indicate statistically significant p-values are highlighted on the x axis.

Figure 4.

Median eat actions for each social information strategy over all environments. Data points on the primary y axis represent the median proportion of the eat action. Data points on the secondary y axis represent the Z-score value from a Mann-Whitney U test comparing, for each environment, the median actions for the two social information strategies presented. Z-scores that indicate statistically significant p-values are highlighted on the x axis.

Figure 5.

Median wait actions for each social information strategy over all environments. Data points on the primary y axis represent the median proportion of the wait action. Data points on the secondary y axis represent the Z score value from a Mann-Whitney U test comparing, for each environment, the median actions for the two social information strategies presented. Z-scores that indicate statistically significant p-values are highlighted on the x axis.

Figure 5.

Median wait actions for each social information strategy over all environments. Data points on the primary y axis represent the median proportion of the wait action. Data points on the secondary y axis represent the Z score value from a Mann-Whitney U test comparing, for each environment, the median actions for the two social information strategies presented. Z-scores that indicate statistically significant p-values are highlighted on the x axis.

The fact that in environment 1, differences in eat and wait actions result in less eating and waiting taking place in social populations, in favor of more movement, also indicates that social agents are willing to risk higher energy expenditure, and are willing to spend less time potentially obtaining energy via eating. This demonstrates that the accommodation of social information leads to a more refined, and ultimately more effective, eating strategy as a result of an increased willingness to move. However, as we can see from the action profile boxplots in Figure 6, the application of eating and waiting actions is drawn from quite a large range in all populations, though the interquartile ranges for all actions do indicate some level of consistency in the application of actions in environment 1.

Figure 6.

Action boxplots for each action, for each social information strategy in environment 1, where there is a 1:1 ratio of positive to negative food resources.

Figure 6.

Action boxplots for each action, for each social information strategy in environment 1, where there is a 1:1 ratio of positive to negative food resources.

The suggestion here is that the significant improvement in task performance seen in social populations over non-social populations in less difficult environments (as in [6]) is a direct result of the behavior differences enabled by the accommodation of social information. However, this does lead us to something of a “chicken and egg” situation; did social information use follow as a result of good foraging (with good foragers acting as useful sources of social information), or did social information use result in the development of good foraging strategies? As no information about plant resources is communicated by social agents (only information about the agents themselves being expressed), it would be sensible to assume that the improved task performance seen by social populations in simpler environments is caused by agents developing behaviors that cause greater exposure to other agents (and therefore more sources of social information), which then leads to improved task performance as a secondary outcome. The fact that movement behavior remains significantly different throughout all tests indicates that some behavioral differences persist despite their providing no improvement in task performance.

### 3.2 Reasons for Moving

It is apparent from Figure 3 that movement behavior for populations permitted to use social information differs significantly from that for non-social populations—this is in contrast to both eating actions (see Figure 4) and waiting actions (see Figure 5), which only show significant differences between social and non-social populations in selected environments. Therefore some analysis of why social agents move is necessary.

One possible indication that increased movement is a direct consequence of increased motivation for agents to interact, and arguably the clearest demonstration of the Allee effect [1, 2], would be if agents were found to aggregate or cluster (i.e., herd or shoal). Figure 7 shows the distributions of the size of agent clusters for each social strategy compared to those for non-social populations, the cluster size being simply the number of other agents an agent has within its visual field. Figure 7 demonstrates an increase in cluster size as environmental difficulty increases, but no clear or significant difference in cluster size between social and non-social populations is observed. The increase in cluster size as environmental difficulty increases is explainable as a consequence of the increased waiting exhibited by all populations; agents move less in difficult environments, resulting in new agents being less likely to move away from the parent agents they are placed close to following a reproduction event. The lack of significant difference in cluster size would likely be a result of moving agents regularly encountering other moving agents due to the density of agents, and of agents clustering around good food resources. Therefore, this result is not totally surprising and leads to the conclusion that the increased movement seen in social populations does not lead to higher levels of aggregation or clustering.

Figure 7.

Distribution of cluster sizes for each social information strategy against the no-social information strategy, for each environment difficulty.

Figure 7.

Distribution of cluster sizes for each social information strategy against the no-social information strategy, for each environment difficulty.

The fact that agents in social populations can actually view other agents enables a second level of analysis regarding why social agents might be motivated to move. When an agent decides to move, rather than wait or eat, they evaluate their preference for each pixel (cell) within their visual field. If a cell contains another agent (either alone, or on a food resource), then a social agent can register an agent view. Non-social populations are blind to other agents, and therefore unable to register agent views. Agent views can be positive (resulting in attraction), negative (resulting in repulsion), or neutral (ultimately not affecting movement behavior). Should non-neutral (positive or negative) agent views be registered, we can conclude that social information is being actively used by agents when moving.

Figure 8 shows the distributions of the number of neutral and non-neutral agent views accumulated by individuals in social populations. It is clear from Figure 8 that for most social strategies, in most contexts, the social information provided by the proximity of other agents is considered to be of little use, and therefore does not affect movement decisions. But it is key to note that for every social strategy in all environments (except environment 9 for age populations), some non-neutral agent views are registered. Sometimes social information is useful, and is therefore used to influence agent behavior. However, the distribution of agent views for populations using age information (Figure (8d)) stands out; unlike the other social strategies, agent views are split relatively evenly between neutral and non-neutral activity. This indicates that some forms of social information can often be useful, and thus worth seeking out.

Figure 8.

Distribution of neutral versus non-neutral agent views for each social strategy over each environment difficulty.

Figure 8.

Distribution of neutral versus non-neutral agent views for each social strategy over each environment difficulty.

Considering how each social strategy was operationalized in the model, it is clear why information about the age of other individuals might prove to be more useful than other types of social information; age is the only unambiguous indicator of success explored here. Presence can help agents in simpler environments decide whether or not to move over to a food resource, but it is unlikely to promote general exploratory movement. Action provides more information, but without additional social information it is difficult for agents to determine whether or not an action (especially eating) is being applied by a successful or reliable individual. Health is better at indicating success, and therefore a more reliable source of information, but is still noisy; young agents are born with full energy levels, and some agents can just be lucky when eating. Age is unambiguous; older agents (especially those who have lived beyond a few epochs) can only have done so by being successful at the task. We see that in the most difficult environment, information about the age of others is not often accumulated, and is never used—this is a result of the environment being so challenging that agents rarely live very long.

Figure 9 assesses whether the non-neutral views accumulated by social populations are perceived to be positive or negative. On drastically different scales across social strategies, we do see a shift from largely positive agent views in presence populations, to largely negative agent views in health and age populations. Action populations demonstrate little preference either way. While not analyzed here, these results do suggest that populations with access to more reliable social information (health and age) are able to be more discerning about whether they wish to move toward another agent, whereas populations with only the presence of other agents available to them have very little cause to be repelled (agents cannot directly interpret the density of agents in an area, and therefore cannot be disinclined to move towards oversaturated resources). But the fact that for social populations, social information can not only be non-neutral but can even be attractive (positive) does provide an explanation for why social populations choose to move more often than non-social populations, even when this doesn't necessarily result in improved task performance; movement may result either from attractive additional stimuli (other agents) or from a motivation to move to seek out other agents, as opposed to just waiting.

Figure 9.

Distribution of positive versus negative agent views for each social strategy over each environment difficulty.

Figure 9.

Distribution of positive versus negative agent views for each social strategy over each environment difficulty.

### 3.3 Behavioral Transitions

From Figure 10 we can see that non-social populations do not exhibit any statistically significant transitions (p < 0.01) between environments in regard to movement behavior. However, statistically significant transitions in movement behavior between environments can be seen in all social populations. For populations using presence information we see this statistically significant transition happen between environments 2 and 3: The transition from primarily eating positive food resources to primarily eating negative food resources also occurs between environments 2 and 3. The association between a statistically significant transition in movement behavior and the transition to primarily consuming negative food resources is also apparent for populations using action information and populations using health information. For health populations it is also interesting to note that statistically significant movement behavioral transitions occur on both occasions when positive food consumption drops below zero. These results demonstrate that movement behavior in social populations is strongly driven by agent task performance; when agents can no longer successfully perform the task, social populations are less inclined to explore their environment in order to seek out new food resources or new sources of social information. In the case of populations using age social information, the only significant transition associated with movement behavior occurs before the transition to non-positive food consumption. The point at which this transition in movement behavior occurs corresponds to a large drop in task performance between environments 2 and 3, demonstrating that movement behavior is still highly sensitive to task performance in age social information populations.

Figure 10.

The median differences between successful and unsuccessful eat actions is presented on the primary y axis along with the boxplots for the move action. The Z-score from Mann-Whitney U tests, which compare the action data for the environment on which a data point falls with that for the previous environment, is presented on the secondary y axis. These Z-scores indicate which transitions in action behavior between previous environments are significant. Z-scores that indicate statistically significant p-values are highlighted on the x axis.

Figure 10.

The median differences between successful and unsuccessful eat actions is presented on the primary y axis along with the boxplots for the move action. The Z-score from Mann-Whitney U tests, which compare the action data for the environment on which a data point falls with that for the previous environment, is presented on the secondary y axis. These Z-scores indicate which transitions in action behavior between previous environments are significant. Z-scores that indicate statistically significant p-values are highlighted on the x axis.

From Figure 2 and from Figures 3, 4, and 5 we can see that agent behavior changes as environments become more difficult. These behavioral changes lead to a reduction in movement and eating, and an increase in waiting. The primary driving force behind the motivation to eat less, move less, and wait more, independent of social information strategy, is that food resources are increasingly likely to be negative in their energy provision, and therefore it makes sense for agents to spend more time conserving their energy by waiting for a positive food source to appear near to them or (in the case of social populations) for an agent whose information suggests they can be trusted to move into their visual field. However, in most cases the increase or decrease in actions as environments become more difficult is not necessarily smooth, this being most apparent with move actions (Figure 3), which for many social information strategies show a sudden reduction in action rather than a steady degradation. It is not clear from the preceding figures whether these changes between environments are statistically significant, nor what is driving these sudden changes when they occur.

In [6] it is shown that task performance (the ability to eat positive food resources more frequently than negative food resources) deteriorates as environments get more difficult—difficulty being defined as the ratio of positive food resources to negative food resources available in the environment. The point at which task performance changes from successful to unsuccessful (the point at which eating actions result in more negative food resources being consumed than positive food resources) varies depending on the social information strategy being tested, but occurs in all scenarios. For no-social and presence populations this transition (zero crossing) occurs between environments 2 and 3, for action populations, between environments 3 and 4, and for both health and age populations, between environments 4 and 5 (though health populations do not permanently cross into negative task performance until after environment 6). Here we assess whether any statistically significant changes in behavior (behavior transitions) could be associated with these zero-crossing events for food consumption.

When considering the total proportion of actions agents dedicate to eating, as seen in Figure 11, we do not see any significant changes in eating behavior that correspond to the point at which task performance makes the transition from predominantly successful to predominantly unsuccessful application of the eat action. Instead, as seen in Figure 4, the median total eat action degrades gradually with task performance. It is also worth noting the extremely large data ranges seen with the total application of each action in the boxplot data in Figure 11. The large interquartile ranges especially show that all populations, social and non-social, are capable of exhibiting very high and very low levels of eating activity. This is in stark contrast to movement, which we can see from Figure 10 has reasonably small interquartile ranges for all population types across all environments, and if anything becomes more consistent as environmental difficulty increases—this being in contrast to the general increase in the range of eat action data, which generally increases as the environment becomes more difficult. Increasingly large data ranges are also seen when we consider the wait action (as seen in Figure 12). Significant transitions in waiting behavior, in all populations other than health ones, do not seem to occur in relation to the transition from positive to negative task performance. These results further indicate that social agents are driven to seek out new sources of social information, but with the caveat that social interactions are likely to result in better task performance; though the fact that social populations move more often than non-social populations even when task performance is poor suggests that social populations still persist in a residual amount of socially motivated movement.

Figure 11.

The median differences between successful and unsuccessful eat actions is presented on the primary y axis along with the boxplots for the eat action. The Z-score from Mann-Whitney U tests, which compare the action data for the environment in which a data point falls with the previous environment, is presented on the secondary y axis. These Z-scores indicate which transitions in action behavior between previous environments are significant. Z-scores that indicate statistically significant p-values are highlighted on the x axis.

Figure 11.

The median differences between successful and unsuccessful eat actions is presented on the primary y axis along with the boxplots for the eat action. The Z-score from Mann-Whitney U tests, which compare the action data for the environment in which a data point falls with the previous environment, is presented on the secondary y axis. These Z-scores indicate which transitions in action behavior between previous environments are significant. Z-scores that indicate statistically significant p-values are highlighted on the x axis.

Figure 12.

The median differences between successful and unsuccessful eat actions is presented on the primary y axis along with the boxplots for the wait action. The Z-score from Mann-Whitney U tests, which compare the action data for the environment in which a data point falls with that in the previous environment, is presented on the secondary y axis. These Z-scores are intended to indicate which transitions in action behavior between previous environments are significant. Z-scores that indicate statistically significant p-values are highlighted on the x axis.

Figure 12.

The median differences between successful and unsuccessful eat actions is presented on the primary y axis along with the boxplots for the wait action. The Z-score from Mann-Whitney U tests, which compare the action data for the environment in which a data point falls with that in the previous environment, is presented on the secondary y axis. These Z-scores are intended to indicate which transitions in action behavior between previous environments are significant. Z-scores that indicate statistically significant p-values are highlighted on the x axis.

## 4 Discussion and Conclusion

In this work we have attempted to address three questions. (1) Does social information lead to increased agent interaction? (2) Is agent interaction, and by extension social information use, dependent on environmental predictability? (3) Do social behaviors persist even when task performance is poor?

Social information transfer is highly prevalent in nature [41], and even the simple presence of other agents has been demonstrated to encourage interesting and novel behaviors in those agents [11], so it is not entirely surprising that the results presented in this work provide strong evidence that social information can lead to interaction-promoting behaviors, namely, movement for the purpose of increasing the probability of agent interaction. We also see social behaviors being favored in the simpler environments tested here. These simpler environments provided agents with a large variety of food resources that could be either negative or positive with equal probability, resulting in a task that was reasonably easy to perform, but also such that it was very difficult for individuals to develop a complete set of categorizations for each food resource's edibility.

Social behaviors favored here are likely to result from social information being more reliable than private information. As environments progressed in difficulty, private information about the edibility of any given food resource became more reliable, as it was increasingly likely that any given food resource was energy reducing and therefore not worth consuming. Any social interaction in later, more difficult environments would still have yielded some benefits, though. In the presence of a food resource in any environment, the presence, actions, health, or age of other local agents could potentially result in a novel or new food resource being evaluated correctly. Despite private information based on the likelihood of edibility encouraging a conservative policy on eating, this new social information could sometimes yield positive results leading to an adaptive advantage over agents who eschew social interaction.

Here we see a continued preference for movement in social information populations compared to non-social populations, even in more difficult environments where task performance in both social and non-social populations was similar. This continued desire to move for the purpose of social interaction was less apparent in later environments, where waiting actions were preferred due to the risk of unnecessary or unrewarding energy expenditure in more difficult environments, but the result was still significantly different from that in non-social cases.

The results presented here add evidence to the idea that a pressure for evolution to adapt to accommodate social information, be it via social information transfer or imitation, is maintained even when social information is either unreliable or risky [20], and therefore suggest that the introduction of simple social information is sufficient, even when decoupled from any within-lifetime learning processes, for the emergence of pro-social behaviors.

Following on from this work, and the work of [6], a number of additional tests are required to fully establish how social information affects agent behavior and to what extent agent behavior is affected by parameters such as the cost of movement, cost of stationarity, population density, proportion of unfit agents replaced at the end of each epoch, and food density, persistence, and energy. As the work currently stands, it is difficult to fully establish whether agents are attracted to certain actions, older individuals, or healthy individuals—the work here simply establishes that the availability of social information can elicit changes of behavior, with the changed behaviors acting as promoters of agent interaction.

## Acknowledgments

Much of this work was completed thanks to PhD funding from the School of Computing and Mathematics, Keele University, UK. We would also like to thank the three anonymous reviewers for their fair and critical comments, and their sound and valuable advice, on earlier versions of this work; this work is undoubtedly stronger as a result of their comments.

## References

References
1
Allee
,
W.
(
1931
).
Animal aggregations: A study in general sociology
.
Chicago
:
University of Chicago Press
.
2
Allee
,
W. C.
(
1939
).
The social life of animals
.
London
:
W. Heinemann
.
3
Blackmore
,
S.
(
1998
).
Imitation and the definition of a meme
.
Journal of Memetics
,
2
(
11
),
159
170
.
4
Blackmore
,
S. J.
(
1999
).
The meme machine
.
Oxford
:
Oxford University Press
.
5
Bonnie
,
K. E.
, &
Earley
,
R. L.
(
2007
).
Expanding the scope for social information use
.
Animal Behaviour
,
74
(
2
),
171
181
.
6
Borg
,
J. M.
, &
Channon
,
A.
(
2017
).
Evolutionary adaptation to social information use without learning
. In
G.
Squillero
&
K.
Sim
(Eds.),
European Conference on the Applications of Evolutionary Computation
(pp.
837
852
).
New York
:
Springer
.
7
Borg
,
J. M.
,
Channon
,
A.
, &
Day
,
C.
(
2011
).
Discovering and maintaining behaviours inaccessible to incremental genetic evolution through transcription errors and cultural transmission
. In
T.
Lenaerts
,
M.
Giacobini
,
H.
Bersini
,
P.
Bourgine
,
M.
Dorigo
, &
R.
Doursat
(Eds.),
Proceedings of the European Conference on Artificial Life 2011
(pp.
102
109
).
Cambridge, MA
:
MIT Press
.
8
Borg
,
J. M.
, &
Channon
,
A. D.
(
2012
).
Testing the variability selection hypothesis—the adoption of social learning in increasingly variable environments
. In
C.
,
D. M.
Bryson
,
C.
Ofria
, &
R. T.
Pennock
(Eds.),
ALIFE 13: The Thirteenth International Conference on the Synthesis and Simulation of Living Systems
(pp.
317
324
).
Cambridge, MA
:
MIT Press
.
9
Boyd
,
R.
, &
Richerson
,
P. J.
(
1985
).
Culture and the evolutionary process
.
Chicago
:
University of Chicago Press
.
10
Bullinaria
,
J. A.
(
2017
).
Imitative and direct learning as interacting factors in life history evolution
.
Artificial Life
,
23
(
3
),
374
405
.
11
Channon
,
A. D.
, &
Damper
,
R.
(
1998
).
The evolutionary emergence of socially intelligent agents
. In
B.
Edmonds
&
K.
Dautenhahn
(Eds.),
Socially situated intelligence: A workshop held at SAB'98, University of Zurich Technical Report
(pp.
41
49
).
12
Dawkins
,
R.
(
1989
).
The selfish gene
(2nd ed.).
Oxford
:
Oxford University Press
.
13
Dennett
,
D. C.
(
1995
).
Darwin's dangerous idea: Evolution and the meanings of life
.
New York
:
Simon and Schuster
.
14
Dugatkin
,
L. A.
, &
Godin
,
J.-G. J.
(
1993
).
Female mate copying in the guppy (Poecilia reticulata): Age-dependent effects
.
Behavioral Ecology
,
4
(
4
),
289
292
.
15
Galef
,
B. G.
, Jr.
(
1996
).
Social enhancement of food preferences in Norway rats: A brief review
. In
C. M.
Heyes
&
B. G.
Galef
(Eds.),
Social learning in animals: The roots of culture
(pp.
49
64
).
San Diego
:
.
16
Galef
,
B. G.
, Jr.
, &
Giraldeau
,
L.-A. G.
(
2001
).
Social influences on foraging in vertebrates: Causal mechanisms and adaptive functions
.
Animal Behaviour
,
61
,
3
15
.
17
Greene
,
C. M.
, &
Stamps
,
J. A.
(
2001
).
Habitat selection at low population densities
.
Ecology
,
82
(
8
),
2091
2100
.
18
Henrich
,
J.
, &
Boyd
,
R.
(
1998
).
The evolution of conformist transmission and the emergence of between-group differences
.
Evolution and Human Behavior
,
19
,
215
241
.
19
Henrich
,
J.
, &
Gil-White
,
F. J.
(
2001
).
The evolution of prestige: Freely conferred deference as a mechanism for enhancing the benefits of cultural transmission
.
Evolution and Human Behavior
,
22
(
3
),
165
196
.
20
Higgs
,
P. G.
(
2000
).
The mimetic transition: A simulation study of the evolution of learning by imitation
.
Proceedings of the Royal Society of London B: Biological Sciences
,
267
(
1450
),
1355
1361
.
21
Hoppitt
,
W.
, &
Laland
,
K. N.
(
2008
).
Social processes influencing learning in animals: A review of the evidence
.
Advances in the Study of Behavior
,
38
,
105
165
.
22
Jolley
,
B. P.
,
Borg
,
J. M.
, &
Channon
,
A.
(
2016
).
Analysis of social learning strategies when discovering and maintaining behaviours inaccessible to incremental genetic evolution
. In
E.
Tuci
,
A.
Giagkos
,
M.
Wilson
, &
J.
Hallan
(Eds.),
From animals to animats 14: 14th International Conference on Simulation of Adaptive Behavior
(pp.
293
304
).
New York
:
Springer
.
23
King
,
A. J.
, &
Cowlishaw
,
G.
(
2007
).
When to use social information: The advantage of large group size in individual decision making
.
Biology Letters
,
3
(
2
),
137
139
.
24
Kirkpatrick
,
M.
, &
Dugatkin
,
L. A.
(
1994
).
Sexual selection and the evolutionary effects of copying mate choice
.
Behavioral Ecology and Sociobiology
,
34
(
6
),
443
449
.
25
Laland
,
K. N.
(
2004
).
Social learning strategies
.
Animal Learning & Behavior
,
32
(
1
),
4
14
.
26
Mitri
,
S.
,
Floreano
,
D.
, &
Keller
,
L.
(
2009
).
The evolution of information suppression in communicating robots with conflicting interests
.
Proceedings of the National Academy of Sciences of the USA
,
106
(
37
),
15786
15790
.
27
Noble
,
J.
,
Todd
,
P. M.
, &
Tuci
,
E.
(
2001
).
Explaining social learning of food preferences without aversions: An evolutionary simulation model of Norway rats
.
Proceedings of the Royal Society of London B: Biological Sciences
,
268
(
1463
),
141
149
.
28
Rendell
,
L.
,
Boyd
,
R.
,
Cownden
,
D.
,
Enquist
,
M.
,
Eriksson
,
K.
,
Feldman
,
M. W.
,
Fogarty
,
L.
,
Ghirlanda
,
S.
,
Lillicrap
,
T.
, &
Laland
,
K. N.
(
2010
).
Why copy others? Insights from the social learning strategies tournament
.
Science
,
328
(
5975
),
208
213
.
29
Rendell
,
L.
,
Fogarty
,
L.
,
Hoppitt
,
W. J.
,
Morgan
,
T. J.
,
Webster
,
M. M.
, &
Laland
,
K. N.
(
2011
).
Cognitive culture: Theoretical and empirical insights into social learning strategies
.
Trends in Cognitive Sciences
,
15
(
2
),
68
76
.
30
Robinson
,
E.
,
Ellis
,
T.
, &
Channon
,
A.
(
2007
).
Neuroevolution of agents capable of reactive and deliberative behaviours in novel and dynamic environments
. In
F.
Almeida e Costa
(Ed.),
Advances in artificial life: Proceedings of the 9th European Conference on Artificial Life
(pp.
345
354
).
New York
:
Springer
.
31
Sergio
,
F.
, &
Newton
,
I.
(
2003
).
Occupancy as a measure of territory quality
.
Journal of Animal Ecology
,
72
(
5
),
857
865
.
32
Stamps
,
J. A.
(
1988
).
Conspecific attraction and aggregation in territorial species
.
The American Naturalist
,
131
(
3
),
329
347
.
33
Stanton
,
A.
, &
Channon
,
A. D.
(
2015
).
Incremental neuroevolution of reactive and deliberative 3D agents
. In
Proceedings of the European Conference on Artificial Life 2015
(pp.
341
348
).
Cambridge, MA
:
MIT Press
.
34
Stephens
,
P. A.
,
Sutherland
,
W. J.
, &
Freckleton
,
R. P.
(
1999
).
What is the Allee effect?
Oikos
,
87
(
1
),
195
190
.
35
Valone
,
T. J.
(
1989
).
Group foraging, public information, and patch estimation
.
Oikos
,
56
(
3
),
357
363
.
36
Valone
,
T. J.
(
2007
).
From eavesdropping on performance to copying the behavior of others: A review of public information use
.
Behavioral Ecology and Sociobiology
,
62
(
1
),
1
14
.
37
van Bergen
,
Y.
,
Coolen
,
I.
, &
Laland
,
K. N.
(
2004
).
Nine-spined sticklebacks exploit the most reliable source when public and private information conflict
.
Proceedings of the Royal Society of London, Series B: Biological Sciences
,
271
(
1542
),
957
962
.
38
van der Post
,
D.
(
2008
).
Learning what to eat: Emerging cultural phenomena in group foragers
.
PhD thesis
,
Utrecht University
.
39
Wagner
,
R. H.
, &
Danchin
,
É.
(
2003
).
Conspecific copying: A general mechanism of social aggregation
.
Animal Behaviour
,
65
,
405
408
.
40
,
H.
, &
Richerson
,
P. J.
(
2009
).
The evolution of conformist social learning can cause population collapse in realistically variable environments
.
Evolution and Human Behavior
,
30
(
4
),
261
273
.
41
Whiten
,
A.
, &
Van Schaik
,
C. P.
(
2007
).
The evolution of animal ‘cultures’ and social intelligence
.
Philosophical Transactions of the Royal Society of London B: Biological Sciences
,
362
(
1480
),
603
620
.
42
Yang
,
S. M.
, &
Meng
,
M.
(
2000
).
An efficient neural network method for real-time motion planning with safety consideration
.
Robotic and Autonomous Systems
,
32
,
115
128
.
43
Yang
,
S. X.
, &
Meng
,
M.
(
2000
).
An efficient neural network approach to dynamic robot motion planning
.
Neural Networks
,
13
(
2
),
143
148
.

### Appendix: Tables of Parameters

Tables A1 to A4 provide an overview of the parameter settings used in this work.

Table A1.
EnVar and plant parameters.
ParameterSettings or range
World size (cells) 700 × 700
Number of plants 500
Plant size (cells) 10 × 10
Negative plant species energy Eneg = −10
Positive plant species energy Epos = 1.0
Eating events until plant is exhausted 200
ParameterSettings or range
World size (cells) 700 × 700
Number of plants 500
Plant size (cells) 10 × 10
Negative plant species energy Eneg = −10
Positive plant species energy Epos = 1.0
Eating events until plant is exhausted 200

Notes. Justifications and rationale: Plant size and quantity were set in order for plants to take up approximately 10% of the world area. During preliminary testing of the system this density of plants ensured plants were a frequently encountered feature of the environment, without being densely packed; agents still often had to search for plants. Further exploration of the results presented here in regard to plant density would be worthwhile, as it would be expected to have a significant effect on movement behavior. The parameter “eating events until plant exhaustion” was set to equal the population size; increasing or decreasing this variable would be expected to affect the proportion of time agents spent eating.

Table A2.
Simulation and population parameters.
ParameterSettings or range
Simulation length (epochs) 100
Epoch length (time steps) 1000
Population size 200
ParameterSettings or range
Simulation length (epochs) 100
Epoch length (time steps) 1000
Population size 200

Notes. Justifications and rationale: Epoch length was set to be approximately the number of time steps required for an agent to move from one corner of the world to the other. Simulation length was based on preliminary testing; both task performance and agent behavior were seen to stabilize for all population types and environments by 100 epochs. Population size was set with regard to computational time to obtain results—larger population sizes had the negative consequences of both longer run times and more densely packed environments. Varying population size would be expected to affect movement behavior, as a higher density of agents would reduce the need to search for other agents.

Table A3.
Agent and evolution parameters.
ParameterSettings or range
Maximum (initial) battery level 100
Stationary energy loss (per time step) 0.1
Movement energy loss (per time step) 0.2
% of population replaced at epoch 10%
Genotype length L = 112 or L = 128
Mutation rate (per locus) pmut = 1/L
Gaussian random noise (mean) μ = 0
Gaussian random noise (standard deviation) σ = 0.01
Crossover Single point
ParameterSettings or range
Maximum (initial) battery level 100
Stationary energy loss (per time step) 0.1
Movement energy loss (per time step) 0.2
% of population replaced at epoch 10%
Genotype length L = 112 or L = 128
Mutation rate (per locus) pmut = 1/L
Gaussian random noise (mean) μ = 0
Gaussian random noise (standard deviation) σ = 0.01
Crossover Single point

Notes. Justifications and rationale: As one agent could not inhabit the same cell as another, agent size was kept small to minimize the need to recalculate agent movement choices. Visual field size was set to be as large as possible with regard to computational time to obtain results. The creation of activity landscapes was computationally expensive and therefore limited the size of visual fields. Larger visual fields would be expected to encourage a higher proportion of movement behavior. Maximum battery level was set in relation to stationary energy loss; the current configuration results in an agent losing all energy within one epoch should they remain static throughout. Movement energy loss was set to be double the stationary energy loss to discourage movement behavior unless selected for. Energy loss could be further explored to better understand the dependence of agent behavior on the cost of behavior.

Table A4.
Neural network (shunting model) parameters.
ParameterSettings or range
Decision network input units i = 7 or i = 8
Decision network hidden units h = 8
Decision network output units o = 2
Maximum Iota value maxI = 15
Minimum Iota value minI = −15
Negative Iota output thresholds θa = −0.5
Positive Iota output thresholds +θa = 0.5
Movement threshold θb = 0.5
ParameterSettings or range
Decision network input units i = 7 or i = 8
Decision network hidden units h = 8
Decision network output units o = 2
Maximum Iota value maxI = 15
Minimum Iota value minI = −15
Negative Iota output thresholds θa = −0.5
Positive Iota output thresholds +θa = 0.5
Movement threshold θb = 0.5

Notes. Justifications and rationale: Hidden layer size was based on preliminary testing—larger hidden layers didn't provide noticeably better performance on the task, but did increase computational time needed to obtain results. The consequence of increasing the maximum Iota value (and decreasing the minimum Iota value) would be to allow activity from a resource to propagate further within an agent's visual field; therefore objects on the edges of visual fields would have greater influence on agent decisions. (It is not anticipated that this would cause a large change in agent behavior.) The maximum and minimum Iota values set here were found to be sufficient for allowing object activation to influence agent decisions. Adjusting thresholds would be expected to affect the likelihood of agent behaviors being applied. Current thresholds do not bias agent decisions in favor of any of the actions.