The relationship between Flocking Behaviour and the Emergence of Leadership

This paper examines the relationship between flocking behaviour and leadership. In order to achieve this aim, we simulate two co-evolving populations of robots: predators and prey. Behavioural and quantitative analysis indicate that a wellstructured hierarchic leadership emerges in the population of predators after the evolution. The emergence of leadership relates to high levels of fitness, so leadership seems to be a winning strategy. We show that the leader role has been assumed by more explorative individuals. Moreover, exploratory behaviours mostly appear when there is a low following behaviour. Therefore, exploratory and following capabilities seem to be complementary both within every replication and within simulations with different perceptual conditions. On the other hand, leadership seems to be a strategy to enable followers to be more explorative.


Introduction
For the modern ethology and biology, groups of animals are autonomous units, enabling members to synchronize some activities, such as collective foraging and coordination in moving (Reebs, 2000).Specifically, the role of Leadership involves different degrees of conflicts.Across species, individuals are more likely to emerge as leaders if they have particular morphological, physiological, or behavioural traits increasing their propensity to act first in all the coordination problems.The consistent correlation between leadership and personality suggests the intriguing possibility that personality differences are maintained in populations, because they foster social coordination (King, et al., 2009).Many theoretical works have focused on how navigational information is exchanged between group members and how such information flow depends on the knowledge held by each member (Couzin, et al., 2005).In one study, the authors have examined the factors contributing to the formation of leadership/followership patterns in flocks of pigeons, focusing on the role of previous navigational experience (Flack, et al., 2012).The results prove that, in order to negotiate joint routes, pigeons make use of a complex decision-making system based on leadership mechanisms.Basically, less experienced birds are likely to follow more experienced conspecifics.All the pigeon groups exhibited a flocking behaviour.Flocking behaviour can be defined as the capability of group's members to follow other individuals drawing those typical "lines", which are called "flocks".These behavioural patterns have been extensively identified by biologists and ethologists in the animal world: researchers tend to make distinctions between the "shoaling" behaviour of fish, the "swarming" behaviour of insects and the "herding" behaviour of land animals.Generally, flocking behaviour is used to identify groups of flying birds, the lines they trace are named "flocks" for this reason (Barnard, 1980).Recently, flocking has been simulated in many computer simulations with the aim of understanding the fundamental mechanisms (Kwasnicka, et al., 2007).Researchers in robotics and agent-based modelling have usually focused on homogeneous groups.In one approach, they have evolved a team of four homogeneous robots for dynamically allocate roles through bodily and communicative interactions (Gigliotta, et al., 2009).In particular, evolved robots show a differentiation in both their communicative and non-communicative behaviours so that only one robot assumes the role of group leader.In another experiment, a group of agents were simulated for the task of reaching a target in a two dimensional environment (Gigliotta, and  Miglino, 2007).Lastly, some researchers have evolved a robot colony to study the possibility for the evolution of leadership patterns (Lee, et al., 2011).In this work each robot has a prearranged social position, such as, leader, follower, and stranger.
In the present paper we discuss an experiment focused on spontaneous leadership emergence mechanisms (namely without any prearrangement of the social roles).The Robots were evolved by the use of Evolutionary Robotics techniques.This experiment has a two-pronged value, one for robotics, one for social science.In robotics: the genetic differentiation of robots' control systems could contribute to build a new generation of autonomous robots with a leadership/followership hierarchic structure needed for navigational tasks in an undiscovered environment.For social sciences and artificial life, it may be possible to answer some interesting questions related to leadership, such as: Is leadership unavoidable for a social decision-making problem?What are the characteristics and skills of a leader?How environmental and individual characteristics affect the emergence of leadership?What is the ratio between the leaders' portion and followers' portion in a group?The final two questions would be: What is the relationship between flocking behaviour and leadership patterns emergence?Does

Experimental Setup
The Task A group of 40 simulated robots live in an environment consisting of a 550cm x 550cm squared arena surrounded by walls.Each robot is inspired by the Khepera Robots bodies, which have circular chassis with a diameter of 5.5cm.The robots' bodies are equipped with visual sensors and two wheels by which the robots move in the environment (see Figure 1).Figure 2 depicts a schematisation of the experimental setup.The environment contains 20 predator robots and 20 prey robots.The only physical difference between the predators and the prey is the colour: blue for predators and green for prey.Both predators and prey are evolved using Evolutionary Robotics methodology (Nolfi, and Floreano, 2000).When a predator robot bumps against walls or against another predator robot, it bounces back in the neighborhood of the contact point facing a new (i.e.randomly chosen) direction.These bumping rules are followed by the prey robots too, with the exception of bumping into the predators.In fact, there are further behavioural difference between predators and prey: whenever a predator's body approaches and touches a prey's body, the prey disappears, meaning that the predator eats it and the prey consequently dies.On the other side, predators cannot die, in this model.Another substantial difference between prey and predators consists of the different fitness function (which will be illustrated in the next paragraph).The vision system of both prey and predators is based on a linear retina made of 9 photoreceptors (R0-R8) that perceive gray scaled colours.The field of view (FOV) of each robot is 90 degrees wide and represents the extent of the observable world that the robot is able to see at any moment.The FOV ranges from -45 degrees to +45 degrees with respect to the face direction (0°), which is the robot's moving direction.In this way, each photoreceptor manages a 10° wide portion of the FOV: the first photoreceptor is associated to a range of [- 45°,-35°] with respect to the direction faced, the second one to [-35°,-25°], and so on.When an object (for instance another robot) is located in front of a photoreceptor (within its own vision angle), it is activated to a value encoding the colour of the object.Perceived colour values are grey-scaled by the retina system and normalised in the range [0,1].Therefore, the prey's green colour activates the photoreceptors at 0.26, which is the normalised value relating to the gray scaled green.The predators' blue colour enables photoreceptors at 0.97.The maximum vision distance for each retina sensor is 55cm.So, if an object is further from a photoreceptor than 55cm, it cannot be detected.

Neural Controller
An Artificial Neural Network (ANN) controls the behaviour of each robot.The neural network consists of 3 layers with 13 neurons in total: each neuron is connected to the other layers with no recurrent connections.This feed forward topology is schematised in Figure 3.The input layer contains 9 neurons which encode the output from the 9 retina's photoreceptors.In other words, input units receive values (normalised in a range between 0 and 1) from the retina's sensors depending on the gray level of the perceived image.The hidden layer consists of 2 units, and the output layers are the controllers for the motor units: output neurons encode the speed of two wheels which enable the robot to move within the environment.The activation of all the network's units are in the range [0,1].
Internal and output neurons are characterised by a sigmoid activation function (logistics).

Artificial evolution
The evolutionary process for the robots is based on a ranking type genetic algorithm.Each individual is identified by a genotype that encodes the neural network's parameters.These encoded parameters represent the synaptic connection weights and biases.Furthermore, initial parameters are randomly generated in the range [-5,+5].Lastly, each parameter is encoded as a sequence of 8 bits.Thus, initially, the environment is populated by a generation of completely "naive" robots (namely, with a randomly generated genome) with no skills on how to move and detect the food sources.In each generation, 40 individuals are inserted into the environment and left to act randomly there.The originality of the algorithm is that individuals are evaluated all together in the environment.In this way, all the robots present in the environment (both prey and predators) in a given moment are characterised by a different genotype that makes them unique in the population (genetic heterogeneity).At the end of each generation, a different ranking and mutation process is applied to both prey and predators, in order to simulate two different species.Each generation is made of 20 epochs.At the beginning of each epoch, every robot starts from random positions.The life time consists of 3,000 time steps.At the end of their life, all the 20 predators are ranked according to the average number of prey eaten in all the epochs.Each of the 4 higher-ranked predators generates 5 offspring which inherit the genotype of their father.The first preserves its father's genotype entirely (elitism) whereas the rest of the offspring' genomes receive a random mutation with a rate of 2%.The total number of new predators (4 x 5=20) populates the next generation.Similarly, the 20 prey robots are ranked separately.All the evolutionary process carries on for 300 generations.All the simulation is repeated for 10 replications (or seeds).The Fitness function is computed differently for predators and prey: when a predator bumps against a prey robot, the prey disappears from the environment (i.e. it is dead) and the predator's fitness score is increased by a value of +1.0.Each predator robot always lives 3,000 time steps.Whereas each prey robot can die at any time, so prey can have a shorter life span than the predators.A prey's fitness is calculated by the number of time steps in which it can survive.

Results
After the evolution, we observed that the predators evolve the ability to run after the prey, and preys evolve the skill of escaping from predators.Moreover we have noticed the emergence of a flocking behaviour between the predators.On the other hand, the prey do not display any specific grouping behaviour, they just tend to explore the environment.The prey simply avoid the predators when they are in the neighbourhood.Average predators' fitness curves reveal a constant trend for the best and average populations, as showed in Figure 4.The steadiness of fitness curves is present for both predators and prey.In spite of fitness constancy, predators and prey improve their skills and performances throughout the generations.This effect has been explained in other related works by the "arms-races" effect (Nolfi, and Floreano, 2006).
In practice, arms-races may emerge in every situation where a co-evolution of two species is present.That is why, in our simulation, fitness curves appear stable.Nevertheless, robots' strategies and skills improve and become more efficient during the evolution: predators become faster to hunt prey and prey become smarter to avoid predators.Another factor that makes predators' fitness curves constant in time, is the fact that, in each generation, only 20 prey can be eaten in total, because the prey will not be replaced in the environment after they die.
To find a single indicator on the fitness reached by robots in each replication, we have calculated the average of average fitness over the last 20 generations (Fitness Indicator) .Apparently there is an unexpected inter-replication variation of fitness.So the first question we have tried to answer is: What is the phenomenon behind the substantial inter-seed variation of average fitness?In order to understand the reason of this variation, we have tried to calculate a static aggregation measure of the predators' populations in the ecological environment.From this point we only consider the predators' population for further analysis, as there are no interesting emerging social behaviours in the prey population for our aim.
The "Aggregation Measure" has been calculated by measuring the distance between each robot and the nearest robot, in each time step, for the last generation's populations.All the time steps and epochs measures have been averaged.The lower values correspond to more aggregation and, vice versa, higher values correspond to less aggregation.By running a correlation between the Fitness Indicator and the Aggregation Measure, the Pearson's Correlation Coefficient between those two series of data is ρ = -0.7,which indicates high reverse-correlation between Fitness and Aggregation.This means that the higher the aggregation, the higher the fitness.In Figure 5 all the series are reported on a plot with the correlation coefficient.
To better understand the mechanisms underpinning the flocking, we have tested each single predator in a reduced environment called "Laboratory".Laboratory is a square with a size of 150cm x 150cm.Firstly, we have inserted a single predator into the environment, and we have plotted the individual trajectory, as illustrated in Figure 6a.Some robots display a small exploratory ability, others a medium exploratory ability, and other robots have a large exploratory ability.By placing the robots side by side in the same environment, we observed the behaviour illustrated in Figure 6b.We have noticed that, almost in every couple, one robot always leads and one robot follows.From these observations it appears that the flocking seems to be regulated by a hierarchy among the predators and this is predetermined in advance by the evolution, this hierarchy is numerically proved by exploration measures illustrated later on.We have supposed the hierarchy is guaranteed by every single trajectory (i.e. by every single exploratory ability).On the other hand, the hierarchy cannot be regulated by the colour, because all the 3 robots have the same colour.Exploiting the information from retina photoreceptors, each robot is able to discern the angle of another robot's movements.In other words, each robot is able to discriminate the arching amplitude of the curvilinear trajectories that another robot is able to draw.By this amplitude information, robots can recognize the hierarchic degree of their partners.Thus, every robot is capable to decide whether to follow or to lead, by using this strategy.The leadership appears to be "relative", namely each leader is not an absolute leader but may be a follower of another robot.
To clarify and support these hypothesis returned by the behavioural analysis, we have developed a series of analytical measures.The first Measure, that we have conceived, is called "Leadership Measure by Vision", which measures each predator's leadership hierarchical rank by exploiting the vision system.We have inserted all the possible predator couples into the Laboratory Environment.Each of the 20 predators have been paired with each of the 19 others.For each sub-test, only 2 robots are present in the environment, in the same time.Then, we have counted in how many time steps each predator sees something, namely how many time steps at least one retina photoreceptor is activated.The hypothesis is that if there are only two individuals in the same environment, the leader will see less than the follower, if there are no other objects.

A) B)
Figure 6: (A) Trajectories of some predator exemplars.They are predator number 27, 29 and 39.Trajectories of predators couples.
In fact, the leader should more likely be at the head of the line whereas the follower should be on the tail.Correlation Coefficient between "Aggregation" and "Leadership Measure by Vision over replications" has returned a ρ = -0.8 that proves a strong reverse-correlation between leadership and aggregation.This correlation is shown in Figure 7.This means that the higher the leadership, the stronger the aggregation in the group.The reverse-correlation appears because of the design of the leadership measure: the higher the leadership measure, the lower the vision value.We can argue that, if the aggregation correlates with the fitness, and fitness correlates with leadership, then the leadership correlates with the fitness.That is, high levels of fitness correlate with high levels of leadership Another interesting issue is in which way is the leadership role connects with the exploratory ability.Predator robots seem to display different exploratory skills.Hence, we have calculated the exploratory ability for each single robot.Each test has been performed on the last generation's population of predators for 20 trials lasting 3,000 time steps.Each value has been averaged over all the trials and reported on the bar-plot The ability to explore has been then related with the ability of the predators to follow other robots in the environment.

ECAL -General Track
To this end we have identified a "Exploratory Ability Measure" which is depicted in Figure 8, dark grey bars and a "Followership Measure".We have measured the exploratory ability of each robot in ecology by counting how many 5.5cm x 5.5cm sized cells each robot visits only once.(we call ecology the evolutionary environment to distinguish it from the smallest laboratory environment) in two different conditions: vision condition, in which the robot can see any other robots in the environment no-vision condition, in which a robot can only see the prey and it is blind to the other presence of the other predators in the environment.This test has been executed, both conditions, on the last generation's predators for 20 trials.We noticed that, the vision condition produces an increase of exploratory abilities, especially in those cases where robots were not good at exploration in the no-vision condition.The increase of exploratory ability has been schematised in Figure 9.As we can observe in the vision condition the less explorative robots became more exploratory.It may indicate that,the ability to follow other predators in the group seems to be a mechanism to make the entire group able to become more exploratory with respect to the situation in which they are alone.This could arguably be an effect of the social relationship in the group.We can then suppose that, in those situations where there are many individuals who are not genetically predisposed to assume exploratory behaviours, leadership may facilitates the group cohesion and performance by increasing the exploration.Another interesting insight derives from measuring the exploratory gap between the no-vision condition and the vision condition, replication by replication.Essentially, we have averaged all the values of the no-vision exploratory ability over replications.
A gap appears between replications (light grey bars): the gap can be regarded as the average ability "to follow" of the robots in one replication. .Indeed, we can consider that, from the no-vision to vision condition, each robot gains an increase of their exploratory abilities, which is directly proportional to the propensity of the robot to follow someone else.If we "isolate" this gap, we acquire a measure of robots' following abilities over replications: the "Followership Measure".By calculating the Pearson's correlation coefficient between Leadership Measure and Followership Measure the value is ρ = -0.79confirming a strong correlation.In Figure 10 this correlation is graphically visualised.This indicates that the stronger is the leadership in a replication, the stronger is the followership in the same replication, as indication of the fact that leadership only emerges where the social group is based on a clear leader-follower organization and someone leads and other follows.The motivation for the flocking emergence is determined by the fact that every predator robot is characterised by a maximum limit of vision distance (55cm), when predators cannot see any prey, they tend to follow another predator rather than doing nothing.This fact is proved in Figure 11 where a chart shows that increasing the vision distance, the following behaviour decreases and vice-versa.The specificity of flocking behaviour is also suggested by the fact that following another predator is notably different than hunting a prey.In fact, when predators move after another con-specific, they do not tend to bump against it, but they just limit themselves to follow keeping a safe distance.Instead, hunting consists of following the prey until the predator reaches it and bumps against it, in order to eat the prey.A careful analysis of the exploratory and following abilities, by means of previous charts, shows another interesting piece of information: the exploratory ability and following ability are reciprocally complementary.This means one ability excludes the other one.For example, in the seed 7, all the predators appear to be explorers rather than followers, whereas in the seed 10 they display an inclination for following rather than exploring.For this reason, we have implemented an analysis of exploration and following abilities depending on different perceptual conditions.In substance, we have re-evolved the robots with different vision conditions, namely by varying the vision distance limit: 13.75cm, 27.5cm, 41.25cm, 55cm, 82.5cm, 165cm and 220cm.We have not been able to sample many more vision conditions because of the elevated computational and time costs of each single evolution.Anyway, the number of completed samples has seemed to be sufficient for the present.The limit of 55cm has been used as the default condition, because it was adopted in the initial evolution.Therefore, we have considered the condition "55cm" as baseline for the comparisons.Again, every simulation has been evolved for 300 generations and for 10 replications in every vision condition.When all the evolutions have been accomplished, we have calculated the average of the "Exploratory Ability Measure over replication" and of the "Followership Measure", for each condition.
The result for the "Exploratory Ability" and "Following Ability" through different vision conditions is depicted in Figures 11.We have graphically interpolated all the points in order to highlight the data trend.

Conclusions
In conclusion, the experiment reported here indicates that in a population of two co-evolving species of robots, with a genetically variable distribution of skills, flocking and leadership are often observed.Although the fitness keeps constant over the generations for the "arms-races" effect, each species' skills appear enhanced at the end of the evolution: predators are better at hunting prey and prey are better at escaping.An inter-replication variation is present in the indicators of predators' fitness and aggregation, which underline a different "social" organisation among the predators that we interpret as leadership, which is mainly due to different initial genetic traits (which are randomly selected).
In replications where there is a strong component of leadership, there seems to be a stronger aggregation.Furthermore, a strongly structured hierarchy appears in the predator's population: the rank of each robot is regulated by the explorative attitude of each robot (namely, the amplitude of movements).Fitness, leadership and followership measures are in strong correlation.
In this way, we guess it is possible to conclude that a group of artificial agents exploit leadership/followership patterns to solve the task of exploring the environment and moving collectively toward the position of the prey.These leadership patterns correlate with higher fitness, which suggests leadership is a winning strategy.In other words, a "peer-topeer" flocking behaviour is not enough to guarantee a smart movement of the group, but the emergence of leadership is needed for achieving better performances.Another interesting result is that all the exploratory individuals do not tend to be good followers and vice versa.This indicates that there is a specialisation of skills in populations, according to different simulated conditions.This suggests a theoretical limit: exploration and following are two complementary skills.
Other interesting future directions could be investigating if the bigger the group size, the smaller the leaders portion.Furthermore, some improvements might be achieved in this simulation by examining, in depth, some of the unclear aspects such as the correlation between leadership emergence and fitness and the relationship between genetic variability and leadership emergence.

Figure 1 :
Figure 1: Schematisation of top and bottom view of the robot chassis.

Figure 2 :
Figure 2: The environment and the robots.

Figure 3 :
Figure 3: The control system of predator and prey robots.

Figure 4 :
Figure 4: Visualisation of the average of all 10 predators' fitness curves, bests (black) and averages (light grey).

Figure 5 :
Figure 5: Correlation between Aggregation Measure and Fitness Indicator.

Figure 7 :
Figure 7: Correlation between Aggregation Measure and Leadership Measure.

Figure 8 :
Figure 8: (A) Exploratory Ability Measure.In the picture there are the values of Replication no.7 and Replication no.10.(B) Exploratory Ability Measure over replications (average).

Figure 9 :
Figure 9: Exploratory Ability Measure in no-vision (dark grey) and vision condition (dark + light grey).The light grey values indicate the Followership Measure.

Figure 11 :
Figure 11: (A) Exploratory Ability through different Distance Vision Limits.(B) Followership Ability through different Distance Vision Limits.