Clearing is a niching method inspired by the principle of assigning the available resources among a niche to a single individual. The clearing procedure supplies these resources only to the best individual of each niche: the winner. So far, its analysis has been focused on experimental approaches that have shown that clearing is a powerful diversity-preserving mechanism. Using rigorous runtime analysis to explain how and why it is a powerful method, we prove that a mutation-based evolutionary algorithm with a large enough population size, and a phenotypic distance function always succeeds in optimising all functions of unitation for small niches in polynomial time, while a genotypic distance function requires exponential time. Finally, we prove that with phenotypic and genotypic distances, clearing is able to find both optima for and several general classes of bimodal functions in polynomial expected time. We use empirical analysis to highlight some of the characteristics that makes it a useful mechanism and to support the theoretical results.
Evolutionary Algorithms (EAs) with elitist selection are suitable to locate the optimum of unimodal functions as they converge to a single solution of the search space. This behaviour is also one of the major difficulties in a population-based EA, the premature convergence toward a suboptimal individual before the fitness landscape is explored properly. Real optimisation problems, however, often lead to multimodal domains and so require the identification of multiple optima, either local or global (Sareni and Krahenbuhl, 1998; Singh and Deb, 2006).
In multimodal optimisation problems, there exist many attractors for which finding a global optimum can become a challenge to any optimisation algorithm. A diverse population can deal with multimodal functions and can explore several hills in the fitness landscape simultaneously, so they can therefore support global exploration and help to locate several local and global optima. The algorithm can offer several good solutions to the user, a feature desirable in multiobjective optimisation. Also, it provides higher chances to find dissimilar individuals and to create good offspring with the possibility of enhancing the performance of other procedures such as crossover (Friedrich et al., 2009).
Diversity-preserving mechanisms provide the ability to visit many and/or different unexplored regions of the search space and generate solutions that differ in various significant ways from those seen before (Gendreau and Potvin, 2010; Lozano and García-Martínez, 2010). Most analyses and comparisons made between diversity-preserving mechanisms are assessed by means of empirical investigations (Chaiyaratana et al., 2007; Ursem, 2002) or theoretical runtime analyses (Jansen and Wegener, 2005; Friedrich et al., 2007; Oliveto and Sudholt, 2014; Oliveto et al., 2014; Gao and Neumann, 2014; Doerr et al., 2016). Both approaches are important to understand how these mechanisms impact the EA runtime and if they enhance the search for obtaining good individuals. These different results imply where/which diversity-preserving mechanism should be used and, perhaps even more importantly, where they should not be used.
One particular option for diversity maintenance are the niching methods; such methods are based on the mechanics of natural ecosystems. A niche can be viewed as a subspace in the environment that can support different types of life. A specie is defined as a group of individuals with similar features capable of interbreeding among themselves but that are unable to breed with individuals outside their group. Species can be defined as similar individuals of a specific niche in terms of similarity metrics. In EAs the term niche is used for the search space domain, and species for the set of individuals with similar characteristics. By analogy, niching methods tend to achieve a natural emergence of niches and species in the search space (Sareni and Krahenbuhl, 1998).
A niching method must be able to form and maintain multiple, diverse, final solutions for an exponential to infinite time period with respect to population size, whether these solutions are of identical fitness or of varying fitness. Such requirement is due to the necessity to distinguish cases where the solutions found represent a new niche or a niche localised earlier (Mahfoud, 1995).
Niching methods have been developed to reduce the effect of genetic drift resulting from the selection operator in standard EAs. They maintain population diversity and permit the EA to investigate many peaks in parallel. On the other hand, they prevent the EA from being trapped in local optima of the search space (Sareni and Krahenbuhl, 1998). In the majority of algorithms, this effect is attained due to the modification of the process of selection of individuals, which takes into account not only the value of the fitness function but also the distribution of individuals in the space of genotypes or phenotypes (Glibovets and Gulayeva, 2013).
Many researchers have suggested methodologies for introducing niche-preserving techniques so that, for each optimum solution, a niche gets formed in the population of an EA. Most of the analyses and comparisons made between niching mechanisms are assessed by means of empirical investigations using benchmark functions (Sareni and Krahenbuhl, 1998; Singh and Deb, 2006). There are examples where empirical investigations are used to support theoretical runtime analyses and close the gap between theory and practice (Friedrich et al., 2009; Oliveto et al., 2014; Oliveto and Zarges, 2015; Covantes Osuna and Sudholt, 2017; Covantes Osuna et al., 2017).
Both fields use artificially designed functions to highlight characteristics of the studied EAs when tackling optimisation problems. They exhibit such properties in a very precise, distinct, and paradigmatic way. Moreover, they can help to develop new ideas for the design of new variants of EAs and other search heuristics. This leads to valuable insights about new algorithms on a solid basis (Jansen, 2013).
Most of the theoretical analyses are made on example functions with a clear and concrete structure so that they are easy to understand. They are defined in a formal way and permit the derivation of theorems and proofs allowing knowledge about EAs to develop in a sound, scientific way. In the case of empirical analyses, most of the results are based on the analysis of more complex example functions and algorithmic frameworks for a specific set of experiments. This approach allows us to explore the general characteristics more easily than in the theoretical field.
Our contribution is to provide a rigorous theoretical runtime analysis for the clearing diversity mechanism, to find out whether the mechanism is able to provide good solutions by means of experiments, and we include theoretical runtime analysis to prove how and why an EA is able to obtain good solutions depending on how the population size, clearing radius, niche capacity, and the dissimilarity measure are chosen.
This article extends a preliminary conference paper (Covantes Osuna and Sudholt, 2017) in the following ways. We extend the theory for large niches by looking into the choice of the population size. While it was known that a population size of is sufficient (Covantes Osuna and Sudholt, 2017), here we show that population sizes of at least are necessary to escape from local optima. The reason is that for smaller population sizes, winners in local optima spawn offspring repeatedly take over the whole population, and this happens before individuals can escape from the optima's basin of attraction. We further extend our analysis to more general classes of example landscapes defined by Jansen and Zarges (2016), showing that clearing is effective across a range of functions with different slopes and optima having different basins of attraction. Finally, we extend the experimental analysis for smaller population sizes of with small and large .
In the remainder of this article, we first present the algorithmic framework in Section 2. The definition of clearing, algorithmic approach, and dissimilarity measures are defined in Section 3. The theoretical analysis is divided in Sections 4 and 5 for small and large niches, respectively. In Section 4 we show how clearing is able to solve, for small niches and the right distance function, all functions of unitation, and in Section 5 we show how the population dynamics with large population size in clearing solves with the most natural distance function: Hamming distance while it fails with a small population size. In Section 6 we show that the analysis made in Section 5 is general enough to be applied into more general function classes. Section 7 contains the experimental results showing how well our theoretical results match with empirical results for the general behaviour of the algorithm and providing a closer look into the impact of the population size on performance. We present our conclusions in Section 8, giving additional insight into the dynamic behaviour of the algorithm.
We focus our analysis on the simplest EA with a finite population called ( + 1) EA (hereinafter, denotes the size of the current population, see Algorithm 1). Our aim is to develop rigorous runtime bounds of the ( + 1) EA with the clearing diversity mechanism. We want to study how diversity helps to escape from some optima. The ( + 1) EA uses random parent selection and elitist selection for survival and has already been investigated by Witt (2006).
We consider functions of unitation , where depends only on the number of 1-bits contained in a string and is always non-negative, i.e., is entirely defined by a function , , where denotes the number of 1-bits in individual . In particular, we consider the bimodal function of unitation called (see Definition 1) for the analysis of large niches. can be seen as a bimodal equivalent of . The fitness landscape consists of two hills with symmetric slopes. In contrast to Friedrich et al. (2009) where an additional fitness value for was added to distinguish between a local optimum and a unique global optimum, we have opted to use the same approach as Oliveto et al. (2014), and leave unchanged since we aim at analysing the global exploration capabilities of a population-based EA.
The fitness of ones increases with more than 1-bits, and the fitness of zeroes increases with less than 1-bits. These sets are referred to as branches. The aim is to find a population containing both optima (see Figure 1).
We analyse the expected time until both optima have been reached. is an ideal benchmark function for clearing as it is simply structured, hence facilitating a theoretical analysis, and it is hard for EAs to find both optima as they have the maximum possible Hamming distance. Its choice further allows comparisons with previous approaches such as in Friedrich et al. (2009) and Oliveto et al. (2014) in the context of diversity-preserving mechanisms.
The ( + 1) EA with no diversity-preserving mechanism (Algorithm 1) has already been analysed for the function. The selection pressure is quite low, nevertheless, the ( + 1) EA is not able to maintain individuals on both branches for a long time. Without any diversification, the whole population of the ( + 1) EA collapses into the branch with probability at least (see Motwani and Raghavan, 1995 for the asymptotic notation) in time ; once the population contains copies of optimum individuals in one of the two branches, it will be necessary to flip all the bits at the same time in order to allow that individual to survive and find both optima on , so the expected optimisation time is (Friedrich et al., 2009, Theorem 1).
Adding other diversity-preserving mechanisms into the ( + 1) EA such as avoiding genotype or phenotype duplicates does not work, the algorithm cannot maintain individuals in both branches, so the population collapses into the branch with a probability at least with an expected optimisation time of and (Friedrich et al., 2009, Theorems 2 and 3, respectively). Deterministic crowding with sufficiently large population is able to reach both optima with high probability in expected time (Friedrich et al., 2009, Theorem 4).
A modified version of fitness sharing is analysed in Friedrich et al. (2009): rather than selecting individuals based on their shared fitness, selection was done on a level of populations. The goal was to select the new population out of the union of all parents and all offspring such that it maximises the overall shared fitness of the population. The drawback of this approach is that all possible size- subsets of this union of size , where is the number of offspring, need to be examined. For large and , this is prohibitive. It is proved that a population-based shared fitness approach with reaches both optima of in expected time (Friedrich et al., 2009, Theorem 5).
In Oliveto et al. (2014), the performance of the original fitness sharing approach is analysed. The analysis showed that using the conventional (phenotypic) sharing approach leads to considerably different behaviours. A population size is not sufficient to find both optima on in polynomial time: with probability the population will reach the same optimum, and from there the expected time to find both optima is (Oliveto et al., 2014, Theorem 1). However, there is still a constant probability to find both optima in polynomial expected time , if the two search points are initialised on different branches, and if these two search points maintain similar fitness values throughout the run (Oliveto et al., 2014, Theorem 2).
With , once the population is close enough to one optimum, individuals descending the branch heading towards the other optimum are accepted. This threshold, that allows successful runs with probability 1, lies further away from the local optimum as the population size increases finding both optima in expected time (Oliveto et al., 2014, Theorem 3). Concerning the effects of the offspring population, increasing the offspring population of a (+) EA, with and cannot guarantee convergence to populations with both optima; that is, depending on , one or both optima can get lost, thus the expected time for finding both optima is (Oliveto et al., 2014, Theorem 4).
Clearing is a niching method inspired by the principle of sharing limited resources within a niche (or subpopulation) of individuals characterised by some similarities. Instead of evenly sharing the available resources among the individuals of a niche, the clearing procedure supplies these resources only to the best individual of each niche: the winner. The winner takes all rather than sharing resources with the other individuals of the same niche as it is done with fitness sharing (Pétrowski, 1996).
As in fitness sharing, the clearing algorithm uses a dissimilarity measure given by a threshold called clearing radius between individuals to determine if they belong to the same niche or not. The basic idea is to preserve the fitness of the individual that has the best fitness (also called dominant individual), while it resets the fitness of all the other individuals of the same niche to zero.1 With such a mechanism, two approaches can be considered. For a given population, the set of winners is unique. The winner and all the individuals that it dominates are then fictitiously removed from the population. Then the algorithm proceeds in the same way with the new population which is then obtained. Thus, the list of all the winners is produced after a certain number of steps.
On the other hand, the population can be dominated by several winners. It is also possible to generalise the clearing algorithm by accepting several winners chosen among the niche capacity (best individuals of each niche defined as the maximum number of winners that a niche can accept). Thus, choosing niching capacities between one and the population size offers intermediate situations between the maximum clearing () and a standard EA ().
Empirical investigations made in Pétrowski (1996; 1997a; 1997b), Sareni and Krahenbuhl (1998), and Singh and Deb (2006) mentioned that clearing surpasses all other niching methods because of its ability to produce a great quantity of new individuals by randomly recombining elements of different niches, controlling this production by resetting the fitness of the poor individuals in each different niche. Furthermore, an elitist strategy prevents the rejection of the best individuals.
We incorporate the clearing method into Algorithm 1, resulting in Algorithm 2. The idea behind Algorithm 2 is: once a population with individuals is generated, an individual is selected and changed according to mutation. A temporary population is created from population and the offspring ; then the fitness of each individual in is updated according to the clearing procedure shown in Algorithm 3.
Each individual is compared with the winner(s) of each niche in order to check if it belongs to a certain niche or not, and to check if it is a winner or if it is cleared. Here is any dissimilarity measure (distance function) between two individuals and of population . Finally, we keep control of the niche capacity defined by . For the sake of clarity, the replacement policy will be the one defined in Witt (2006): the individuals with best fitness are selected (set of winners) and individuals coming from the new generation are preferred if their fitness values are at least as good as the current ones (novelty is rewarded).
Finally, as dissimilarity measures, we have considered genotypic or Hamming distance, defined as the number of bits that have different values in and : , and phenotypic (usually defined as Euclidean distance between two phenotypes). As is a function of unitation, we have adopted the same approach as in previous work (Friedrich et al., 2009; Oliveto et al., 2014) for the phenotypic distance function, allowing the distance function to depend on the number of ones: where and denote the number of 1-bits in individual and , respectively.
4 Small Niches
In this section, we prove that the ( + 1) EA with phenotypic clearing and a small niche capacity is not only able to achieve both optima of but is also able to optimise all functions of unitation with a large enough population, while genotypic clearing fails in achieving such a task (hereinafter, we will refer as phenotypic or genotypic clearing to Algorithm 3 with phenotypic or genotypic distance function, respectively).
4.1 Phenotypic Clearing
First, it is necessary to define a very important property of clearing, which is its capacity of preventing the rejection of the best individuals in the ( + 1) EA, and once is defined large enough, clearing and the population size pressure will always optimise any function of unitation.
Note that on functions of unitation all search points with the same number of ones have the same fitness, and for phenotypic clearing with clearing radius all search points with the same number of ones form a niche. We refer to the set of all search points with ones as niche . In order to find an optimum for any function of unitation, it is sufficient to have all niches , for , being present in the population.
In the ( + 1) EA with phenotypic clearing with , and , a niche can contain only winners with ones. The condition on ensures that the population is large enough to store individuals from all possible niches.
Consider the ( + 1) EA with phenotypic clearing with , and on any function of unitation. Then winners are never removed from the population, i.e., if is a winner then .
After the first evaluation with clearing, individuals dominated by other individuals are cleared and the dominant individuals are declared as winners. Cleared individuals are removed from the population when new winners are created and occupy new niches. Once an individual becomes a winner, it can be removed only if the size of the population is not large enough to maintain it, as the worst winner is removed if a new winner reaches a new better niche. Since there are at most niches, each having at most winners, if , then there must be a cleared individual among the parents and offspring considered for deletion at the end of the generation. Thus, a cleared individual will be deleted, so winners cannot be removed from the population.
The behaviour described above means, that with the defined parameters and sufficiently large to occupy all the niches, we have enough conditions for the furthest individuals (individuals with the minimum and maximum number of ones in the population) to reach the opposite edges. Now that we know that a winner cannot be removed from the population by Lemma 2, it is just a matter of finding the expected time until and are found.
Because of the elitist approach of the ( + 1) EA, winners will never be replaced if we assume a large enough population size. In particular, the minimum (maximum) number of ones of any search point in the population will never increase (decrease). We first estimate the expected time until the two most extreme search points and are being found, using arguments similar to the well-known fitness-level method (Wegener, 2002).
Let be a function of unitation and , and . Then, the expected time for finding the search points and with the ( + 1) EA with phenotypic clearing on is .
Where the summation is known as the harmonic number and satisfies . Adding the same time for finding proves the claim.
Once the search points and have been found, we can focus on the time required for the algorithm until all intermediate niches are discovered.
Let be any function of unitation, , and , and assume that the search points and are contained in the population. Then, the expected time until all niches are found with the ( + 1) EA with phenotypic clearing on is .
According to Lemma 2 and the elitist approach of ( + 1) EA, winners will never be replaced if we assume a large enough population size and by assumption we already have found both search points and .
Let be a function of unitation and , and . Then, the expected optimisation time of the ( + 1) EA with phenotype clearing on is .
Now that we have defined and proved all conditions where the algorithm is able to maintain every winner in the population (Lemma 2), to find the extreme search points (Lemma 3) and intermediate niches (Lemma 4) of the function , we can conclude that the total time required to optimise the function of unitation is .
4.2 Genotypic Clearing
In the case of genotypic clearing with , the ( + 1) EA behaves like the diversity-preserving mechanism called no genotype duplicates. The ( + 1) EA with no genotype duplicates rejects the new offspring if the genotype is already contained in the population. The same happens for the ( + 1) EA with genotypic clearing and if the population is initialised with mutually different genotypes (which happens with probability at least ). In other words, conditional on the population being initialised with mutually different search points, both algorithms are identical. In Friedrich et al. (2009, Theorem 2), it was proved that the ( + 1) EA with no genotype duplicates and is not powerful enough to explore the landscape and can be easily trapped in one optimum of . Adapting Friedrich et al. (2009, Theorem 2) to the goal of finding both optima and noting that for the considered yields the following.
The probability that the ( + 1) EA with genotypic clearing, and finds both optima on in time is at most . The expected time for finding both optima is .
As mentioned before, the use of a proper distance is really important in the context of clearing. In our case, we use phenotypic distance for functions of unitation, which has been proved to provide more significant information at the time it is required to define small differences (in our case small niches) among individuals in a population, so the use of that knowledge can be taken into consideration at the time the algorithm is set up. Otherwise, if there is no more knowledge related to the specifics of the problem, genotypic clearing can be used but with larger niches as shown in the following section.
5 Large Niches
While small niches work with phenotypic clearing, Corollary 6 showed that with genotypic clearing small niches are ineffective. This makes sense as for phenotypic clearing with a niche with ones covers search points, whereas a niche in genotypic clearing with covers only one search point. In this section we turn our attention to larger niches, where we will prove that cleared search points are likely to spread, move, and climb down a branch.
We first present general insights into these population dynamics with clearing. These results capture the behaviour of the population in the presence of only one winning genotype (of which there may be copies). We estimate the time until in this situation the population evolves a search point of Hamming distance from said winner, for any , or for another winner to emerge (for example, in case an individual of better fitness than is found).
These time bounds are very general as they are independent of the fitness function. This is possible since, assuming the winners are fixed at , all other search points within the clearing radius receive a fitness of 0 and hence are subject to a random walk. We demonstrate the usefulness of our general method by an application to with a clearing radius of , where all winners are copies of either or . The results hold both for genotypic clearing and phenotypic clearing as the phenotypic distance of any point to (, resp.) equals the Hamming distance of to (, resp.).
5.1 Large Population Dynamics with Clearing
Before proving the lemma, let us make sense of this formula. Ignore the term for the moment and consider the formula . Note that is the average distance to the winner in . If the population has spread such that it has reached an average distance of , then the expected change would be . Moreover, a smaller average distance will give a positive drift (expected value in the decrease of the distance after a single function evaluation) and an average distance larger than will give a negative drift. This makes sense as a search point performing an independent random walk will attain an equilibrium state around Hamming distance from .
The term reflects the fact that losers in the population do not evolve in complete isolation. The population always contains copies of that may create offspring and may prevent the population from venturing far away from . In other words, there is a constant influx of search points descending from winners . As the term indicates, this effect grows with , but (as we will see later) it can be mitigated by setting the population size sufficiently large.
The potential allows us to conclude when the population has reached a search point of distance at least from . The following lemma gives a sufficient condition.
If contains copies of and then must contain at least one individual with .
There are at most individuals different from . By the pigeon-hole principle, at least one of them must have at least distance from .
In order to bound the time for reaching a high potential given in Lemma 8, we will use the following drift theorem, a straightforward extension of the variable drift theorem (Johannsen, 2010) toward reaching any state smaller than some threshold . It can be derived with simple adaptations to the proof in Rowe and Sudholt (2014).
The following lemma now gives an upper bound on the first hitting time (the random variable that denotes the first point in time to reach a certain point) of a search point with distance at least to the winner .
Let be the current population of the ( + 1) EA with genotypic clearing and on any fitness function such that contains copies of a unique winner and for all . For any , if then the expected time until a search point with is found, or a winner different from is created, is .
We pessimistically assume that no other winner is created and estimate the first hitting time of a search point with distance at least . As can only increase by at most in one step, is an upper bound on the maximum potential that can be achieved in the generation where a distance of is reached or exceeded for the first time.
In order to apply drift analysis, we define a distance function that describes how close the algorithm is to reaching a population where a distance was reached. We consider the random walk induced by , stopped as soon as a Hamming distance of at least from is reached. Due to our definition of , the random walk only attains values in as required by the variable drift theorem.
The minimum threshold for contains a factor of . The reason is that the fraction of winners in the population needs to be small enough to allow the population to escape from the vicinity of . The population size hence needs to grow proportionally to the number of winners the population is allowed to store.
Note that the restriction is necessary in Lemma 10. Individuals evolving within the clearing radius, but at a distance larger than to will be driven back toward . If is significantly larger than , we conjecture that the expected time for reaching a distance of at least from becomes exponential in .
5.2 Upper Bound for
Consider the ( + 1) EA with genotypic or phenotypic clearing, and on with a population containing copies of (). Then the expected time until a search point with at least (at most) ones is found is .
The expected time for the ( + 1) EA with genotypic or phenotypic clearing, , and finding both optima on is .
By Corollary 11, the expected time until a search point on the opposite branch is created is . Since the best individual on the opposite branch is a winner in its own niche, it will never be removed. This allows the population to climb this branch as well. Repeating the arguments from the first paragraph of this proof, the expected time until the second optimum is found is at most . Adding up all expected times proves the claim.
One limitation of Theorem 12 is the steep requirement on the population size: . The condition on was chosen to ensure a positive drift of the potential for all populations that have not reached distance yet, including the most pessimistic scenario of all losers having distance to . Such a scenario is unlikely as we will see in Sections 7.1 and 7.2 where experiments suggest that the population tends to spread out, covering a broad range of distances. With such spread, a distance of can be reached with a much smaller potential than that indicated by Lemma 8. We conjecture that the ( + 1) EA with clearing is still efficient on if . However, proving this theoretically may require new arguments on the distribution of the winners and losers inside the population.
5.3 On the Choice of the Population Size for
To get further insights into what population sizes are necessary, we show in the following that the ( + 1) EA with clearing becomes inefficient on if is too small, that is, smaller than . The reason is as follows: assume that the population contains only a single optimum , and further individuals that are well within a niche of size surrounding . Due to clearing, the population will always contain a copy of . Hence, there is a constant influx of individuals that are offspring, or, more generally, recent descendants of . We refer to these individuals informally as young; a rigorous notation will be provided in the proof of Theorem 13. Intuitively, young individuals are similar to , and thus are likely to produce further offspring that are also young, that is, similar to when chosen as parents.
We will show in the following that if the population size is small, young individuals will frequently take over the whole population, creating a population where all individuals are similar to . This takeover happens much faster than the time the algorithm needs to evolve a lineage that can reach a Hamming distance to the optimum.
The following theorem shows that if the population size is too small, the ( + 1) EA is unable to escape from one local optimum, assuming that it starts with a population of search points that have recently evolved from said optimum.
Consider the ( + 1) EA with genotypic or phenotypic clearing on with , and , starting with a population containing only search points that have evolved from one optimum within the last generations. Then the probability that both optima are found within time is .
The following lemma describes a stochastic process that we will use in the proof of Theorem 13 to model the number of “young” individuals over time. We are interested in the first hitting time of state as this is the first point in time where young individuals have taken over the whole population of size . The transition probabilities for states reflect the evolution of a fixed-size population containing two species (young and old in our case): in each step one individual is selected for reproduction, and another individual is selected for replacement. If they stem from the same species, the size of both species remains the same. But if they stem from different species, the size of the first species can increase or decrease by 1, with equal probability.
This is similar to the Moran process in population genetics (Ewens, 2004, Section 3.4) which ends when one species has evolved to fixation (i.e., has taken over the whole population) or extinction. Our process differs as state 1 is reflecting, hence extinction of young individuals is impossible. Notably, we will show that, compared to the original Moran process, the expected time for the process to end is larger by a factor of order . Other variants of the Moran process have also appeared in different related contexts such as the analysis of Genetic Algorithms (Lemma 6 in Dang et al., 2016) and the analysis of the compact Genetic Algorithm (Lemma 7 in Sudholt and Witt, 2016). The following lemma gives asymptotically tight bounds on the time young individuals need to evolve to fixation.
For the second statement, we use standard arguments on independent phases. By Markov's inequality, the probability that takeover takes longer than is at most . Since the upper bound holds for any , we can iterate this argument times. Then the probability that we do not have a takeover in steps (using ) is .
Now we prove that the time required to reach a new niche with is larger than the time required for “young” individuals to take over the population. In other words, once a winner is found and assigned to an optimum, with a small , the time for a takeover is shorter than the required time to find a new niche. This will imply that the algorithm needs superpolynomial time to escape from the influence of the winner and consequently it needs superpolynomial time to find the opposite optimum.
We analyse the dynamics within the population by means of so-called family trees. The analysis of EAs with family trees has been introduced by Witt (2006) for the analysis of the ( + 1) EA. According to Witt, a family tree is a directed acyclic graph whose nodes represent individuals and edges represent direct parent-child relations created by a mutation-based EAs. After initialisation, for every initial individual there is a family tree containing only . We say that is the root of the family tree . Afterwards, whenever the algorithm chooses an individual as parent and creates an offspring out of , a new node representing is added to along with an edge from to . That way, contains all descendants from obtained by direct and indirect mutations.
There may be family trees containing only individuals that have been deleted from the current population. As individuals survive in every selection, at least one tree is guaranteed to grow. A subtree of a family tree is, again, a family tree. A (directed) path within a family tree from to represents a sequence of mutations creating out of . The number of edges on a longest path from the root to a leaf determines the depth of .
Witt (2006) showed how to use family trees to derive lower bounds on the optimisation time of mutation-based EAs. Suppose that after some time the depth of a family tree is still small. Then typically the leaves are still quite similar to the root. Here we make use of Lemma 1 in Sudholt (2009) (which is an adaptation from Lemma 2 and proof of Theorem 4 in Witt, 2006) to show that the individuals in are still concentrated around . If the distance from to all optima is not too small, then it is unlikely that an optimum has been found after steps.
For the ( + 1) EA with or without clearing, let be an individual entering the population in some generation . The probability that within the following generations some emerges with is .
Lemma 1 in Sudholt (2009) applies to (+) EAs without clearing. We recap Witt's basic proof idea to make the article self-contained and also to convince the reader why the result also applies to the ( + 1) EA with clearing.
The analysis is divided in two parts. In the first part, it is shown that family trees are unlikely to be very deep. Since every individual is chosen as parent with probability , the expected length of a path in the family tree after generations is bounded by . Large deviations from this expectation are unlikely. Lemma 2 in Witt (2006) shows that the probability that a family tree has depth at least is . This argument relies only on the fact that parents are chosen uniformly at random, which also holds for the ( + 1) EA with clearing.
For family trees whose depth is bounded by , all path lengths are bounded by . Each path corresponds to a sequence of standard bit mutations, and the Hamming distance between any two search points on the same path can be bounded by the number of bits flipped in all mutations that lead from one search point to the other.
By applying Chernoff bounds (see Motwani and Raghavan, 1995) with respect to the upper bound on the expectation instead of the expectation itself (cf. Witt, 2006, page 75), we obtain that the probability of an individual of Hamming distance at least to emerging on a particular path is at most . Taking the union bound over all possible paths in the family tree still gives a failure probability of . Adding the failure probabilities from both parts proves the claim.
Now, Lemma 15 implies the following corollary.
The probability that, starting from a search point , within generations the ( + 1) EA with clearing evolves a lineage that reaches Hamming distance at least to its founder is .
By assumption, all individuals in the population are descendants of individuals with genotype , and this property will be maintained over time. This means that every individual in the population at time will have an ancestor that has genotype (our notion of ancestor and descendant includes the individual itself). Tracing back 's ancestry, let be the most recent generation where an ancestor of has genotype . Then we define the age of as . Informally, the age describes how much time a search point has had to evolve differences from the genotype .
Note that the age of itself is always 0 and as the population always contains a winner , it always contains at least one individual of age 0.
Now assume that a new search point is created with . If has age at most then there exists a lineage from a copy of to that has emerged in at most generations. This corresponds to the event described in Corollary 16, and by said corollary the probability of this event happening is at most . Taking the union bound over all family trees (of which there are at most in every generation) and the first generations, the probability that such a lineage does emerge in any family tree and at any point in time within the considered time span is still bounded by .
We now show using Lemma 14 that it is very unlikely that individuals with age larger than emerge. We say that a search point is -young if it has genotype or if its most recent ancestor with genotype was born during or after generation . Otherwise, is called -old. We omit the parameter “” whenever the context is obvious. A key observation is that youth is inheritable: if a young search point is chosen as parent, then the offspring is young as well. If an old search point is chosen as parent, then the offspring is old as well, unless mutation turns the offspring into a copy of .
Let be the number of young individuals in the population at time , and pessimistically ignore the fact that old individuals may create young individuals through lucky mutations. Then, in order to increase the number of young individuals, it is necessary and sufficient to choose a young individual as parent (probability ) and to select an old parent for replacement. The probability of the latter is as there are old parents and the individual to be removed is chosen uniformly at random among individuals whose fitness is cleared. Hence, for , . Similarly, the number of old individuals increases if and only if an old individual is chosen as parent (probability ) and a young individual is chosen for replacement (probability ), hence for we have . Otherwise, . Note that since the winner is young and will never be removed. This matches the Markov chain analysed in Lemma 14.
Now consider a generation where all individuals in the population have ages at most . By assumption, this property is true for the initial population. At time , the population contains at least one -young individual: the winner . By Lemma 14, with probability at least , within the next generations, using the condition , the population will reach a state where , that is, all individuals are -young. Assuming this does happen, let denote the first point in time where this happens. Then at time all individuals have ages at most , and we can iterate the above arguments with instead of .
Each such iteration carries a failure probability of at most . Taking the union bound over failure probabilities over the first generations yields that the probability of an individual of age larger than emerging is only .
Adding failure probabilities and completes the proof.
We conjecture that a population size of is sufficient to optimise in expected time , that is, that the conditions in Theorem 12 can be improved.
6 Generalisation to Other Example Landscapes
Note that, in contrast to previous analyses of fitness sharing (Friedrich et al., 2009; Oliveto et al., 2014), our analysis of the clearing mechanism does not make use of the specific fitness values of . The main argument of how to escape from one local optimum depends only on the size of its basin of attraction. Our results therefore easily extend to more general function classes that can be optimised by leaving a basin of attraction of width at most .
We consider more general classes of examples landscapes introduced by Jansen and Zarges (2016) addressing the need for suitable benchmark functions for the theoretical analysis of evolutionary algorithms on multimodal functions. Such benchmark functions allow the control of different features such as the number of peaks (defined by their position), their slope and their height (provided in an indirect way), while still enabling a theoretical analysis. Since this benchmark setting is defined in the search space and it uses the Hamming distance between two bit strings, it matches perfectly with the current investigation.
Jansen and Zarges (2016) define their notion of a landscape as the number of peaks and the definition of the peaks (numbered ) where the -th peak is defined by its position , its slope , and its offset . The general idea is that the fitness value of a search point depends on peaks in its vicinity. The main objective for any optimisation algorithm operating in this landscape is to identify those peaks: a highest peak in exact optimisation or a collection of peaks in multimodal optimisation. A peak has been identified or reached if the Hamming distance of a search point and a peak is . Since we are considering maximisation, it is more convenient to consider instead.
There are three different fitness functions used to deal with multiple peaks in Jansen and Zarges (2016); we consider the two most interesting function classes and defined in the following. We only consider genotypic clearing in the following as phenotypic clearing only makes sense for functions of unitation.
Let and peaks be given, then
, called the nearest peak function,
, called the weighted nearest peak function,
The nearest peak function, , has the fitness of a search point determined by the closest peak that determines the slope and the offset . In cases where there are multiple that minimise , should additionally maximise . If there is still not a unique individual, a peak is selected uniformly at random from those that minimise and those that maximise .
The weighted nearest peak function, , takes the height of peaks into account. It uses the peak that yields the largest value to determine the function value. The bigger the height of the peak, the bigger its influence on the search space in comparison to smaller peaks.
6.1 Nearest Peak Functions
We first argue that our results easily generalise to nearest peak functions with two complementary peaks , arbitrary slopes , and arbitrary offsets . The generalisation from peaks as for to peaks is straightforward: we can swap the meaning of zeros and ones for any selection of bits without changing the behaviour of the algorithm; hence, the ( + 1) EA with clearing will show the same stochastic behaviour on peaks as well as on arbitrary peaks . As for , if only one peak has been found, the basin of attraction of the other peak is found once a search point with Hamming distance at least to is generated. If the clearing radius is set to , the ( + 1) EA with clearing will create a new niche, and from there it is easy to reach the complementary optimum . In fact, our analyses from Section 5 never exploited the exact fitness values of ; we only used information about basins of attraction, and that it is easy to locate peaks via hill climbing. We conclude our findings in the following corollary.
The expected time for the ( + 1) EA with genotypic clearing, , , and finding both peaks on any nearest peak function with two complementary peaks is .
If , and , and the ( + 1) EA starts with a population containing only search points that have evolved from one optimum within the last generations, the probability that both optima are found within time is .
6.2 Weighted Nearest Peak Functions
For things are different: the larger the peak, the larger is its influence area of the search space in comparison to smaller peaks and thus will have a larger basin of attraction. These asymmetric variants with suboptimal peaks with smaller basin of attraction and peaks with larger basin of attraction are similar to the analysis made in Section 6.1, as long as the parameter is set as the maximum distance between the peaks necessary to form as many niches as there are peaks in the solution, and the restriction of Lemma 10 is met, the same analysis can be applied for this instance of the family of landscapes benchmark.
Along with our previous upper bound on from Theorem 12 it is easy to show the following result for a large class of weighted nearest peak functions .
Note that in case there is only one global optimum: the fitter of the two peaks. Then the respective condition (where the left-hand side inequality is true) implies that the basin of attraction of the less fit peak must be bounded by . If this condition is not satisfied, the function is deceptive as the majority of the search space leads towards a non-optimal local optimum.
The proof is similar to the proof of Theorem 12. Assume without loss of generality that . Using the same arguments as in said proof (with straightforward changes to the fitness-level calculations), the ( + 1) EA finds one peak in expected time . If this is , the ( + 1) EA still needs to find . By the same arguments as in the proof of Theorem 12, the ( + 1) EA's population will contain copies of in expected time . Applying Lemma 10 with yields that the expected time to find a search point with Hamming distance at least to is . Since , by Theorem 2 in Jansen and Zarges (2016), is outside the basin of attraction of . As it is also a winner in a new niche, this new niche will never be removed, and can be reached by hill climbing on a -like slope from . By previous arguments, will then be found in expected time .
As a final remark, the analysis has shown that it is possible to escape of the basin of attraction of the higher peak with ; this does not mean that the analysis cannot be applied to . We need to remember that the current investigation considers a distance because any distance larger than may lead to a exponential expected time in for reaching a distance of at least from . One way to avoid this limitation is by dividing the distance into several niches by setting the parameter properly. In this analysis we just considered the population dynamics and its ability of escaping a basin of attraction of at most or escaping from a niche with radius at most but it may be possible to generalise the population dynamics for more than two niches with sizes by changing/adapting our definition of the potential function. For the time being we rely on experiments in Section 7.3 to show that the population can jump from niches with allowing to find both optimum in different variants of from the classes of example functions and leave the generalisation of the population dynamics for future theoretical work.
The experimental approach is focused on the analysis of the ( + 1) EA and is divided in three experimental frameworks. Section 7.1 is focused on an empirical analysis for the general behaviour of the algorithm, the relationship between the parameters , , and , and how these parameters can be set. The main objective is to compare our asymptotic theoretical results with empirical data for concrete parameter values.
For the second empirical analysis (Section 7.2), we focus our attention on the population size for small () and large () problem sizes. The objective is to observe whether smaller population sizes than are capable of optimising and compare if the quadratic dependence on is an artefact of our approach. Also, we compare two different forms of initialising the population: the standard uniform random initialisation against a biased initialisation where the whole population is initialised with copies of one peak ( for ). Biased initialisation is used in order to observe how clearing is able to escape from a local optimum and how fast it is compared to a random initialisation.
Finally, for the third analysis (Section 7.3), we show that it is possible to escape from different basin of attractions for weighted peak functions with two peaks in cases where the two peaks are not complementary, but have different Hamming distances.
7.1 General Behaviour
We are interested in observing if the ( + 1) EA with clearing is able to find both optima on , so we consider exponentially increasing population sizes for just one size of and perform 100 runs with different settings of parameters and , so for this experimental framework, we have defined , with phenotypic distance since it has been proven that this distance metric works for both cases, small and large niches (when the genotypic distance is used it will be explicitly mentioned).
Since we are interested in proving how good/bad clearing is, we define the following outcomes and stopping criteria for each run. Success: the run is stopped if the population contains both and in the population. Failure: once the run has reached 1 million of generations and one of the two (or both) optima are not contained in the population. All the results are shown in Table 1.
For small values of and , with sufficiently many individuals , every individual can create its own niche, and since only one individual is allowed to be the winner, the individuals are spread in the search space reaching both optima with 1.0 success rate. In this scenario, since we are allowing sufficiently many individuals in the population, individuals can be initialised in any of both branches, climbing down the branch reaching the opposite branch and reaching the other extreme optima as shown in Figure 2 (in this case we show only the behaviour of the population with , with have the same behaviour).
The previous experiment setup confirms what is mentioned in Section 4.1: with a small clearing radius, niche capacity, and large enough population size, the algorithm is able to exhaustively explore the search space without losing the progress reached so far. The population size provides enough pressure to optimise . In this scenario, the small differences between individuals allow the algorithm to discriminate between the two branches or optima—this forces having individuals on both branches or occupying all niches supporting the statement of Theorem 5. Individuals with the same phenotype may have a large Hamming distance, creating winners with the same fitness (as proved by Corollary 6 in Section 4.2).
It is mentioned in Pétrowski (1996) that while the value of the niche capacity approaches the size of the population, the clearing effect vanishes and the search becomes a standard EA. This effect is verified in the present experimental approach. For a large and , one branch takes over, removing the individuals on the other branch reducing the performance of the algorithm. In order to avoid this, it is necessary to define to occupy all the winners slots and create new winners in other niches (Theorem 5) or increase the clearing radius to in order to let more individuals participate in the niche. A reduced niche capacity seems to have a better effect exploring both branches.
Now that theory and practice have shown that a small clearing radius, niche capacity and large enough population size are able to optimise , and in order to avoid the take over of a certain branch due to a large niche capacity it is necessary to either have: (1) a large enough population, or (2) to increase the clearing radius. For (1) we already have defined and proved that one way to overcome this scenario is to define according to Theorem 5.
In the case of large niches (2), with it is possible to divide the search space in fewer niches. Here the individuals have the opportunity to move, change inside the niche, reach other niches allowing the movement between branches, reaching the opposite optimum. Since it is possible to reach other niches, defining the niche capacity will allow to have more winners in each niche but will still allow movement inside the niche.
For example, with , and , the algorithm is able to reach both optima with at least 0.97 success rate. In Figure 3, the effect of can be seen with sufficiently many individuals. With restrictive niche capacity (Figure 3a), the population is scattered in the search space but when the niche capacity is increased, the spread is reduced as we allow more individuals to be part of each niche (Figures 3b and 3c). This behaviour can be generalised and is more evident for larger values of .
The theoretical results described in Section 5 are confirmed by the previous experimental results: a large enough population is necessary in order to fill the positions of the winner with winners, then force those winners with the rest of the population to be subject to a random walk, where it is just necessary for at least one individual to reach the next niche as mentioned in Section 5.1. In the case of , it is after the repetition of moving, climbing down through different niches for a certain period of time when some individual is able to reach both optima as mentioned in Theorem 12 and confirmed by the experiments.
Now we have defined the conditions where the algorithm is able to optimise , we can set up the parameters in a more informed/smart way. With it is possible to optimise if and are chosen appropriately. For example, with (as the minimum distance required to distinguish between one branch and the other), and , the algorithm is able to optimise because there is always an individual moving around that is able to reach a new niche (Figure 4), and finally achieve 1.0 success rate.
7.2 Population Size
In this section, we address the limitation of Theorem 12 related to the steep requirement of the population size: . As observed from Section 7.1, experiments suggest that a smaller population size is able to optimise . So for the analysis of the population size we have considered the population size in order to observe what is the minimum population size below the threshold able to optimise . With , and for with phenotypic clearing, we report the average of generations of 100 runs; the run is stopped if both optima have been found or the algorithm has reached a maximum of 1 million generations—this is enough time for the algorithms to converge on one or both optima.
Figure 5a shows the average number of generations among 100 runs with . Even for the average runtime is below the 1 million threshold, hence some of the runs were able to find both optima on in fewer than 1 million generations. The reason for this high average runtime is because once both individuals have reached one optimum; it will be one winner, and one loser subjected to a random walk until it gets replaced by an offspring of the winner. This process will continue until the loser reaches a Hamming distance of from the optimum to escape the basin of attraction. Once this is achieved, it is just necessary for the individual to climb the other branch.
Most importantly, all population sizes in the interval are able to optimise ; this experimental setting shows that with a smaller population size for a relatively small the algorithm is able to optimise . Another interesting characteristic of the algorithm is its capacity for escaping from a local optimum.
For , in Figure 5b it is more evident that with a small population size, it is not possible to escape from the basin of attraction of a peak, and the takeover happens before the population has the chance to evolve a distance of at least confirming the theoretical arguments described in Section 5.3. Once the population size is increased, the population is able to escape the basin of attraction. The most interesting result shown in Figure 5 is that even with a population size the algorithm is able to find both optima on (even if runs require more than 1 million generations), indicating that the quadratic dependence on in , is an artefact of our approach.
Finally, for larger sizes and it can be seen that biased initialisation is noticeably faster than random initialisation, and as the population grows the difference between the means grows. One reason could be simply because one peak has already been found, and the algorithm only needs to find the remaining peak.
7.3 Escaping from Different Basins of Attraction
Finally, in this section we show that the runtime analysis used in Section 5.1, and used to prove the theoretical analysis on the general classes of example landscapes functions in Section 6 can be used for weighted peak functions with two peaks of different Hamming distances. For simplicity we restrict our attention to equal slopes and heights: and .
We can simplify this class of functions by using that the ( + 1) EA is unbiased as defined by Lehre and Witt (2012): simply speaking, the algorithm treats all bit values and all bit positions in the same way. Hence we can assume without loss of generality that . We can further imagine shuffling all bits such that , which again does not change the stochastic behaviour of the ( + 1) EA. Then all functions with and are covered by choosing and from the set . As can be seen the peaks and can be as close as and , or as far as and .
It contrast to the simple setting of where makes most sense, in this more general setting it is necessary to define the clearing radius according to the Hamming distance between peaks. In particular, the following conditions should be satisfied.
as otherwise one peak is contained in the clearing radius around the other peak,
as otherwise a niche can contain the majority of search points in the search space, leading to potentially exponential times to escape from the basin of attraction of a local optimum if , and
as this is the minimum distance that distinguishes the two peaks.
In the following, we study two different choices of : the maximum value that satisfied the above constraints, and the minimum feasible value, . These two choices allow us to investigate the effect of choosing large or small niches in this setting.
We use genotypic clearing with and we make use of the results from Section 7.2 to define as a population size able to optimise for large . We report the average number of generations of 100 runs, with the same stopping criterion: both optima have been found or the algorithm has reached a maximum of 1 million generations.
For the case of large clearing radius, , Figure 6a shows that it is possible to find both optima efficiently across the whole range of . For random initialisation there are hardly any performance differences, except for a drop in the runtime when the two peaks get very close. For the case of biased populations we see differences by a small constant factor: the closer the peaks, the more difficult it is to escape (or find the new niche) since it requires to flip a specific number of bits to find the other optima. But as the two peaks move away both initialisation methods seem to behave the same indicating that the arguments used in Sections 5.1 and 6 reflect correctly how the algorithm behaves.
Figure 6b shows that with the smallest feasible clearing radius the algorithm is still able to find both optima for all , but the average runtime for biased initialisation is much higher compared to . From observing the actual population dynamics during a run, it seems that the reason for this high number of generations is because several niches are created around both peaks, i.e., once a peak has been found (and a niche is formed around the peak), the population spreads out by forming many niches between and .
In the case of biased initialisation it is necessary to jump between specific niches to reach the opposite peak, or make several jumps between different niches in order to escape from its basin of attraction, which leads to this high number of generations.
The presented theoretical and empirical investigation has shown that clearing possesses desirable and powerful characteristics. We have used rigorous theoretical analysis related to its ability to explore the landscape in two cases, small and large niches, and provide an insight into the behaviour of this diversity-preserving mechanism.
In the case of small niches, we have proved that clearing can exhaustively explore the landscape when the proper distance and parameters like clearing radius, niche capacity, and population size are set. Also, we have proved that clearing is powerful enough to optimise all functions of unitation. In the case of large niches, clearing has been proved to be as strong as other diversity-preserving mechanisms like deterministic crowding and fitness sharing since it is able to find both optima of the test function .
The analysis made has shown that our results can be easily extended to more general classes of examples landscapes. The analysis done for can easily be applied to different classes of bimodal problems using arguments based on how to escape the basin of attraction of one local optimum. We demonstrated this for functions with two complementary peaks and asymmetric variants of , consisting of a suboptimal peak with a smaller basin of attraction and an optimal peak with a larger basin of attraction.
Our experimental results suggest that the same efficient performance also applies to bimodal functions where the two peaks have varying Hamming distances. Here clearing is able to escape from local optima with different basins of attractions by moving/jumping between niches formed by the clearing radius. Defining as the smallest possible value that allows us to distinguish between peaks creates several small niches, forcing the individuals in the population to make several jumps between niches until an individual can reach the basin of attraction of the other peak. This means that the algorithm requires more generations to find both peaks. But if is defined as the maximum feasible value, , the ( + 1) EA is faster and remarkably robust with respect to the Hamming distances between the two peaks. Nevertheless, both approaches allow the population to escape from different basin of attractions.
It remains an open problem to theoretically analyse the population dynamics of clearing with more than two niches and to prove rigorously that clearing is effective across a much broader range of problems, including problems with more than two peaks. This involves obtaining more detailed insights into the dynamics of the population, including the distribution and evolution of the losers across multiple niches.
The authors would like to thank Carsten Witt for his advice and comments on the analysis of the choice of the population size for , and the anonymous reviewers of this manuscript and the previous FOGA publication for their many valuable suggestions. We also thank the Consejo Nacional de Ciencia y Tecnología—CONACYT (the Mexican National Council for Science and Technology) for the financial support under the grant no. 409151 and registration no. 264342. The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement no. 618091 (SAGE).
We tacitly assume that all fitness values are larger than 0 for simplicity. In case of a fitness function with negative fitness values, we can change clearing to reset fitness to , where is the minimum fitness value of , such that all reset individuals are worse than any other individuals.