Many factors influence the evolvability of populations, and this article illustrates how intrinsic mortality (death induced through internal factors) in an evolving population contributes favorably to evolvability on a fixed deceptive fitness landscape. We test for evolvability using the hierarchical if-and-only-if (h-iff) function as a deceptive fitness landscape together with a steady state genetic algorithm (SSGA) with a variable mutation rate and indiscriminate intrinsic mortality rate. The mutation rate and the intrinsic mortality rate display a relationship for finding the global maximum. This relationship was also found when implementing the same deceptive fitness landscape in a spatial model consisting of an evolving population. We also compared the performance of the optimal mutation and mortality rate with a state-of-the-art evolutionary algorithm called age-fitness Pareto optimization (AFPO) and show how the two approaches traverse the h-iff landscape differently. Our results indicate that the intrinsic mortality rate and mutation rate induce random genetic drift that allows a population to efficiently traverse a deceptive fitness landscape. This article gives an overview of how intrinsic mortality influences the evolvability of a population. It thereby supports the premise that programmed death of individuals could have a beneficial effect on the evolvability of the entire population.
In classical Darwinism, senescence—the deterioration of function with age—directly opposes the evolutionary advantage longevity can have. The older an individual can become, the more offspring it could potentially produce. Hence, the personal fitness of an individual is directly increased by living longer. Nature is riddled with mechanisms that seem to restrict longevity, with senescence being prevalent in a multitude of species. Though the individual benefit is usually related to fitness, Darwin  already noted that longevity is likely a product of the complex interactions between a species and its environment. Therefore, could longevity itself somehow be a determinant for the evolutionary trajectories we see in nature?
Many octopus species are semelparous, reproducing only once in their lifetime. This is an observation already mentioned as far back as Aristotle's History of Animals : They “live young and die fast” [29, p. 1592]. The process of senescence in Enteroctopus dofleini, for example, is regulated by secretions from an endocrine gland that normally causes death by starvation after reproduction . In this case, the octopus suddenly stops foraging and instead takes care of its eggs and hatchlings, followed by the eventual death of the parent octopus. However, on simply removing this endocrine gland, octopuses seemingly live significantly longer than usual, and they can reproduce more than once . Senescence in octopuses is particularly elusive, and the possible advantage of this type of senescence might be caused by various phenotypic traits and selection pressures. Does a short life have an evolutionary advantage, or is the decreased life span a by-product of the mechanism inhibiting foraging behavior, which enables the protection of offspring with the dire side effect of mortality?
Other animals, such as some salmon, undergo a similar process of senescence: dying after having laid eggs. Some spiders are cannibals and kill their male mate after copulation (cannibalism is also a feature exhibited by some octopuses). Elephants run out of teeth, a form of mechanical senescence, while some turtles express negligible senescence (not showing aging symptoms). Naked mole rats can become significantly older than other rodents. Artificial selection of Drosophila can allow them to live 50% longer after a few generations , and the proteins DAF-2 and DAF-16 directly regulate life span in C. elegans . Furthermore, it has been shown that long-lived yeast mutants are outcompeted by short-lived wild types . Longevity and aging thus seem to emerge differently across species. But what is the evolutionary value of senescence, if there is any? Considering recent publications by Kowald and Kirkwood  and by Goldsmith  that discuss whether aging is programmed or not, it is relevant to test whether mortality poses any benefit for evolutionary algorithms (EAs) that could support any of the existing theories on aging.
One of the theories of senescence suggests that senescence can promote the evolvability of a population . Evolvability is the population's ability to traverse the fitness landscape without passing through nonfunctional regions [15, 33] and we define it as follows:
Definition (Evolvability): The ability of a population to create adaptive genetic diversity across generations.
Hereby, we consider: (1) the entire population (or gene pool) instead of the individual, (2) genetic diversity to be adaptive (leading to a better-fit solution, in contrast to non-adaptive genetic diversity), and (3) the measurement of evolvability over many generations instead of one. The definition of evolvability used here is different from some existing measures of evolvability [1, 20, 37]. The main difference in our definition is that evolvability should be measured over many generations. The ability of the individual or a population to produce better-adapted offspring or more diverse individuals in one generation is less important than the ability of the population to keep producing better offspring across generations. A population that can produce fitter individuals in one generation might be unable to traverse a local valley in the fitness landscape that needs to be crossed to find a better solution. Therefore, we analyze the efficiency at which a population is able to traverse the fitness landscape over generations as a proxy of evolvability.1
Intrinsic mortality leading to evolvability has been mostly discussed hypothetically . Notably, Herrera et al.  investigated the evolvability of a population of agents in a rapidly changing environment. They show that a terminal age allows the population to better continuously adapt to its environment. In addition,  showed that extinction events can lead to a better evolvable EA. However, in contrast to the work presented here, extinction events were discriminative and kept certain elites in the population. Although evolution is usually seen as including incremental improvements over generations, some solutions might require evolutionary steps that make individuals worse than their ancestors. This less elitist approach could enable progeny to find a solution in the search space that is more distant, and perhaps ultimately more efficient, than their ancestral solutions. As a testbed for this potential leap, we used a deceptive fitness function—an adjusted version of the hierarchical if-and-only-if (h-iff) function —and a genetic algorithm (GA) to simulate an evolving population. This landscape is deceptive, since it contains local optima that draw an EA away from the best solutions [8, 10]. We show that on this test bed, intrinsic mortality alters the evolvability of the population. In addition, a spatial model that inherently contains extrinsic mortality factors is used to show that the added intrinsic mortality rate still changes the evolvability of the population. Though too much mortality is detrimental, depending on the mutation rate of the population, a certain mortality rate actually increases the evolvability of the evolving populations.
We have described in  how this intrinsic mortality parameter can be beneficial for finding solutions on the h-iff function. This article extends the previous one by including additional results on the h-iff landscape, and by a comparison on how a population evolves using our method and age-fitness Pareto optimization (AFPO ). It furthermore highlights the relevance of these results to nature, supporting the theory of programmed death inducing evolvability. The next subsections of the introduction give an overview of the existing theories of senescence and whether mortality can be programmed (Sections 1.1 and 1.2). It follows up with a brief illustration of how intrinsic mortality could promote evolvability (Section 1.3). The experimental setups of the evolutionary algorithms and the spatial model (Section 2) and their results (Section 3) are subsequently presented. Afterwards, the results, and their consequences, are discussed in Section 4. Overall, this article not only illustrates the potential evolutionary advantage of senescence in nature, but also demonstrates how this concept could be beneficial in evolutionary computation.
1.1 Summary of Theories of Senescence
Mortality is a fundamental component of natural systems that is caused either by intrinsic factors (senescence) or by extrinsic factors such as predation, disease, and accidents. It initially seemed that aging is an evolutionary disadvantage for individuals, since an individual's personal fitness is lowered when it dies from internal mechanisms. There are, however, several theories explaining the cause and function of this biological phenomenon in spite of its being a direct disadvantage. Darwin already mentioned in the sixth edition of On the Origin of Species (not in older editions) that longevity is related to the scale of organization, expenditure, and general activity of organisms, which has likely been determined by natural selection . Although having an evolutionary disadvantage for the individual, it may have several advantages for the maintenance of the species. Furthermore, Weismann has claimed that aging is determined by the “needs of the species” [39, p. 9], which are subject to the same mechanical process of regulation as other structures and functions of organisms. Weismann also proposed that this intrinsic mortality makes a population more evolvable, since there is a higher turnover rate of individuals.
An alternative theory by Medawar  proposes that aging could arise merely from the simple neglect of selective pressure on older organisms. Older organisms are, by chance, more likely to have encountered mortality-inducing factors that limit their life span. Considering a steady probability of death for each individual in a specific population, a survivor curve can be created displaying how many individuals in a specific age group are alive (Figure 1). Consider a population of 100 individuals where in every month there is a specific chance for individuals to die—a fixed probability of death, or mortality rate. This probability shapes the age range that evolutionary selection can act on. In the case of a 20% mortality chance every month, the range of selection is quite low, whereas this range is quadrupled when the chance is only 5% (Figure 1). The number of older individuals in a population depends on this rate. Since there are fewer older individuals, as displayed by the survivor curve, the selection pressure on old age is reduced. An accumulation of mutations can therefore creep in, which results in the senescence of the individuals. Genes beneficial in early life would therefore have a higher selective advantage and thereby a higher chance to propagate themselves into the next generation. However, mere accumulation of mutations as an explanation for senescence is difficult to hold for most species, since it has been shown that single genes that cause aging have been conserved throughout different species over evolutionary time .
The effect of certain genes can also change during the lifetime of an individual. According to the theory known as antagonistic pleiotropy , a gene can have a pleiotropic effect by promoting reproductive success and survival early in life, while being detrimental later in life. Through this process, evolutionary biologists have argued that this inherent tradeoff makes it difficult for natural selection to evolve old age in the first place. Although the scientific literature contains a ubiquity of examples of pleiotropic genes, it is more difficult to see how mutations in genes can improve the personal fitness early in life, while having a deleterious effect later in life. These genes are sometimes referred to as “putative” disease alleles [4, p. 4], since evidence that these types of alleles really have a benefit early in life still needs to be acquired.
Another alternative theory to the accumulation of mutations and antagonistic pleiotropy is the disposable soma hypothesis proposed by Kirkwood , which is one of the dominating theories today . In the disposable soma theory, the body of an individual organism allocates limited resources to various cellular processes, and needs to make compromises between its metabolism, reproduction, repair, and maintenance functions. For example, a population only focusing on repair can be outcompeted by a population that instead spends more energy on growth and reproduction. Combined with Medawar's survival curve, wherein death can also be caused by extrinsic factors such as predation, this would suggest that maintenance and repair are also of lesser importance later in life, since the probability of an individual reaching old age by chance is already low. Not allocating any resources to the repair of an organism with increasing age could thus lead to an organism's deterioration.
Other, more recent theories consider the potential altruistic effect of senescence, through which aging can be beneficial for coping with a changing environment [16, 26, 43]. In this case, it has been artificially shown that a terminal age is beneficial for a population in rapidly changing environments that necessitate adaptive changes in the genome. Similarly, a resulting benefit from senescence, or intrinsic mortality, is the reduction of overconsumption of environmental resources that gives a selective advantage for intrinsic mortality . In addition, Lehman et al.  showed that extinction events could lead to a better evolvable evolutionary algorithm, though in this case, the extinction events were discriminative and kept certain elites in the population. Considering the previous results, from an optimization perspective, it seems that senescence, or simply mortality, can be beneficial for a population with regard to evolvability and altruistic aging.
The evolvability theory of senescence claims that senescence increases the evolvability of a population . We think mortality may aid evolvability in two ways:
When individuals die, a higher turnover rate of new individuals arises in the population (proposed by Weismann ).
Mortality reduces selective pressure on the best individuals in the population and thereby decreases convergence, promoting diversification.
For the first reason, a larger number of individuals that can live during a specific period leads to a larger proportion of acquired genetic adaptations by a population. In other words, more individuals that have been evaluated in the environment yield more individual fitness results from potentially differing phenotypes. Mortal populations thus contain a higher turnover rate of individuals than do immortal populations. The second argument implies that if older and fit individuals outcompete younger, slightly less fit individuals, the population can get stuck in a local optimum, or be in a state that is considered less evolvable.
The actual mechanisms of senescence would most likely be a combination of the theories of senescence that have been described. However, if we consider the population (or gene pool2) as a whole instead of thinking about the benefits for the individual, there is no reason for individually detrimental phenotypic traits not to pose an evolutionary advantage. For instance, if mutations are the main factor driving senescence, mutations also drive evolution due to the introduction of new variations of genes in the population. In contrast, a non-mutating population with immortal individuals would reside in a zero-evolvability state . The evolution of complex organisms can thus be a compromise between evolvability of the species and personal benefit to the individual .
The antagonistic pleiotropy theory might surely be an explanation for senescence, though if senescence turns out to be beneficial for a population, this antagonistic effect would actually be an altruistic effect. If the personal fitness of an individual could be prolonged by adjusting the self-repair energy expenditure as mentioned by the disposable soma theory, there are evolutionary pressures toward better self-repair mechanisms. However, the lack of self-repair might also simply lead to more mutations, making a population more evolvable. Alternatively, the lack of self-repair mechanisms could lead to senescence, which could be advantageous as well. Essentially, if there is a selective pressure toward senescence, we can speak of programmed death.
1.2 Programmed versus Non-programmed Senescence
The theories of senescence may be categorized into non-programmed and programmed. Non-programmed theories of senescence include the mutation accumulation , antagonistic pleiotropy , and disposable-soma theories [17, 18]. Programmed-aging theories of evolvability support evolvability [13, 26, 39] and altruistic aging [16, 40, 43]. It seems that supporters of the non-programmed theories generally exclude programmed theories [18, 32], whereas the programmed-senescence supporters do not specifically exclude non-programmed theories.
The consequences of the theories considering life span as an advantage or disadvantage are summarized in Figure 2, adapted from Goldsmith , where increased age is considered to be a disadvantage in line 4 (supporting programmed senescence), but not in lines 1 (classical, no negative effect), 2 (mutation accumulation), and 3 (antagonistic pleiotropy and disposable soma). In , Figure 2 has been criticized in that line 3 does not represent the antagonistic-pleiotropy or the disposable-soma theory well. The parameters of these theories lead to a specific average life span that is optimal; hence, mortality is an emergent factor. However, if there were a gene that, at no cost, would improve the life span of the organism, it would have an advantage for both the antagonistic-pleiotropy and disposable-soma theories, but not for the programmed-aging theories. Age itself is not the disadvantage, but rather the disadvantage is the product of the pleiotropic gene, or the tradeoff between somata. Therefore, the distinction made by Goldsmith  is still valid, but should be taken lightly. The main useful distinction is the evolutionary advantage of the process of senescence—or the evolutionary disadvantage of long life—which line 4 displays. We would therefore like to emphasize that we view our results as representing line 4.
1.3 Why Intrinsic Mortality Can Promote Evolvability
In order to understand how mortality can induce evolvability, it is useful to clarify some concepts from evolutionary dynamics that led to the experiments in this article. Considering any population of individuals at carrying capacity in an environment, and stating that the mortality rate is fixed in this population, we find that the mutation rate greatly influences the types of genes in the population and the resulting stable attractor space in a quasi-species equilibrium. As explained by Nowak , when considering a sequence space of a specific gene, there can be several optima in this space . As depicted in Figure 3, if the average mutation rate u is below a specific critical value u1, the stable (robust) state of the gene in the population will end up in a narrow peak of the fitness landscape. When the mutation rate is at a value between u1 and u2, the narrow peak becomes an unstable region in the sequence space for the population, and the population will converge to the broader sequence space with a lower maximum fitness value. If the mutation rate is in turn increased to be higher than u2, there will be no stable state, and the sequence space in which the population resides will be random. However, if genes in a population of individuals already reside in the broader less-fit state, how can they traverse the sequence space to end up in the narrow peak that is the better-fit solution? Traversing this fitness landscape would either require an individual to drastically mutate into that region, or a population to gradually move to the region through genetic drift. Nowak's mutation-rate threshold values are, moreover, only valid for a population of mortals. If immortality could occur, an immortal individual residing in the narrow peak would always stay there (since it could not be outcompeted), and eventually its offspring would also have a chance to occupy the narrow region, no matter how high the mutation rate was. Mortality in Nowak's model is thus a requirement. Therefore, we consider δ to be the mortality rate, and we claim [35, 36] that there exist mortality rate thresholds δ1 and δ2 similar to the mutation rate thresholds (Figure 3). This is the initial hypothesis that has formed the premise of why mortality promotes evolvability.
Using h-iff as the difficult-to-solve deceptive fitness landscape on both a steady-state genetic algorithm (SSGA) and a spatial model can help us understand how this relationship influences the evolvability of a population. The SSGA is used as an abstract model to view the general effects of mortality on the evolutionary progression on this deceptive fitness landscape. Additionally, as described by Werfel et al. , spatial models can elucidate aspects of mortality that occur in natural systems. The spatial model—which contains an inherent extrinsic mortality rate emerging from local competition—is used to see whether the influence of intrinsic mortality affects evolvability when an emergent extrinsic factor for mortality is also present.
The experiments are divided between a benchmark optimization implementation using a SSGA and an agent-based grid model.3 In both approaches, the fitness value of an individual is calculated based on the h-iff function. The selection and deletion operators in the spatial model are inherent properties of the interactions of the individuals with their environment, while the SSGA uses a random selection operator. Including an extrinsic mortality mechanism in the spatial model allows us to (1) test whether the mutation rate can alter the stable region in the sequence space of the genomes as explained by , and (2) investigate if an additional intrinsic mortality rate influences the evolvability. The SSGA uses both 64-bit and 128-bit genomes. In addition, the SSGA is compared with AFPO, and we discuss how each implementation traverses the fitness landscape differently. The experiments investigate whether mortality alters the evolutionary progression of a population of individuals containing binary genomes and whether this enables the population to traverse the state space landscape more efficiently.
2.1 Hierarchical If-and-Only-If
There are many binary fitness landscapes that have been designed to be convoluted and deceptive, such as the Chuang f1 , the Royal Road , or the hierarchical if-and-only-if (h-iff) function . The h-iff function is used throughout this article to represent our deceptive fitness landscape.4 This function is chosen because it creates a fractal deceptive fitness landscape and it can be used to evaluate the performance of evolutionary algorithms. In h-iff, a binary genome is evaluated based on self-similarity. One can check for self-similarity in the genome in multiple layers by initially checking the similarity of a pair of bits across the genome, continuing in the next layer by checking a pair of two bits, followed by checking a pair of four bits, and so on. In each layer, a fitness value can be ascribed to the self-similarity score of the genome. This score is usually derived from the number of self-similar parts in the genome and the layer depth that is being checked.
As an example, Table 1 illustrates how one can derive a fitness value from a 16-bit genome that results in four layers on which to check for self-similarity. Note that in the original implementation of h-iff, a null bit was possible in the genome, which resulted in an additional fifth layer. This null possibility has been omitted in this article to increase the computational efficiency and ease the visualization of genetic change over time. The omission of this possibility makes the evolutionary progression easier to visualize by plotting the fitness value against the number of ones in the genome (Figure 4). The gray area in Figure 4 illustrates the possible fitness values an individual can achieve when having a certain number of ones in a genome containing 64 bits.
|1||1||0||0||1||1||1||1||1||6 · 1||6|
|2||0||0||1||1||2 · 2||4|
|3||0||1||1 · 4||4|
|4||0||0 · 8||0|
|1||1||0||0||1||1||1||1||1||6 · 1||6|
|2||0||0||1||1||2 · 2||4|
|3||0||1||1 · 4||4|
|4||0||0 · 8||0|
For h-iff, there are two potential global maxima regardless of the length of the genome. One global maximum contains a bit string of only ones, while the other contains only zeros. The maximum fitness of a 64-bit individual with either only zeros or only ones is 192, the value of the global maxima in 64-bit h-iff. However, when half of the genome is composed of zeros and the other half of ones, the fitness value of that particular individual ranges somewhere between 4 and 160, depending on the specific order of the bits. In between the extremes of the global optima, there are many local optima and one can generally state that there is a local optimum between each two higher optima. This makes the landscape inherently fractal and deceptive. To understand how different sets of genomes correspond to fitness values based on the landscape produced by h-iff, Figure 5 illustrates how four genomes are located on the fitness landscape of Figure 4, using the explanation in Table 1. A score for self-similarity in this illustration is simply denoted by a red color. The area of the fields of the table that are colored red directly translates into a fitness value. In this example, the best-fit genome is a bit string of only zeros.
Considering a binary genome that consists of 64 bits, there are a total of 264 possible configurations of a genome. This large search space in turn makes it difficult for algorithms to evaluate this binary function. Moreover, for one experiment we also implemented 128-bit genomes to illustrate that the global maximum can still be found in a larger search space. This 128-bit implementation amounts to a search space of 2128 total possible configurations, which is around 20 orders of magnitude larger than 64-bit genomes.
2.2 Steady State Genetic Algorithm
The steady state genetic algorithm (SSGA) with a mortality rate implements a 64-bit genome composed of either ones or zeros, which are randomly initialized. Genes in the genome are mutated with a probability given by the mutation rate. Note that mutating a gene will randomly assign a bit of 1 or 0, so the gene swaps a bit with half the probability of the mutation rate in mutation events. Thus, a mutation rate of 0.1 means a bit is mutated with a 10% probability but changed with only 5% probability. This implementation ensures that a mutation rate of 1.0 does not produce offspring with the complementary bit string of their parent's genome, but rather an entirely random set of bits.
Each SSGA iteration is as follows: (1) a random individual is chosen, (2) the chosen genome is copied, mutated, and evaluated, (3) the new genome is compared with a random individual in the existing population, and (4) the new genome replaces this second individual if the fitness for the new genome is higher. For a population size n, a generation consists of n iterations. After each generation, individuals are independently checked for deletion with a probability given by the mortality rate. The population was logged after each generation. To isolate the effect of the mutation rate, no crossover was implemented.
We performed three experiments using the SSGA. The first determined the correlation between the mortality rate and the mutation rate on 64-bit h-iff, the second on 128-bit h-iff, and the third compared the SSGA with mortality with age-fitness Pareto optimization (AFPO) . For the first experiment, 20 simulations of 100,000 generations and a population size of 50 individuals with different values for the mortality rate and mutation rate were conducted. A mutation rate sweep from 0.0 to 1.0 was done, changing the mutation rate approximately exponentially. A similar sweep was done for the mortality rate, but the mortality rates 0.64 and 1.0 have been excluded from the results, since these values led to early extinction of the population. For the second experiment, 20 simulations of 100,000 generations and a population size of 50 were conducted with various mutation rates and mortality rates. In the last experiment we compared the optimal mutation or mortality rate found by doing this sweep with the optimal mutation rate of AFPO on 64-bit h-iff, again using a population size of 50. The optimal rates were determined by evaluating which combination of parameters led to the highest frequency of the global solution being found per evolutionary run.
To briefly illustrate, AFPO starts by initializing a population of random individuals. In the initial generation (generation 0), each individual has an age of 0. The age values of the individuals increment by one after each generation in AFPO. When offspring are created, they will have the same age value as their parents. After evaluating the individuals, only the fittest individual in each age category is kept, while the rest of the individuals are discarded. After initialization in AFPO, the iteration (one generation) procedure is as follows: (1) create offspring and mutate them (the number of offspring is the population size −1), (2) increment the age of all individuals, (3) add one random individual with age 0, (4) evaluate all individuals, and (5) remove all Pareto-dominated individuals from the population. So in essence, an individual can outcompete and replace another individual only if its fitness is higher and its age is lower, so that it dominates on both fronts. The number of individuals created in one generation is equal to the maximum population size variable, which has been set to 50.
For this last experiment using AFPO, we ran 200 simulations using optimal parameters and kept track of how many evolutionary runs found the global maximum with 64-bit h-iff. To compare AFPO with the mortality rate SSGA, the optimal mutation rate for AFPO was determined. This was done by running 20 simulations for mutation rates of 0.025, 0.05, 0.1, 0.2, and 0.4 for 50,000 generations. The mutation rate that led to the highest maximum fitness values across multiple evolutionary runs was 0.1, and the mutation rate of the experiments was therefore subsequently set to 0.1. The optimal mortality rate of the SSGA was tuned using the optimal mutation rate 0.1 of AFPO. For this tuning step we ran 20 evolutionary runs, varying the mortality rate between 0.01, 0.025, 0.05, 0.1, and 0.2. These preliminary results determined that a mortality rate of 0.05 was optimal (one can determine from Table 2 in Section 3.1 that 0.05 was a decent mortality rate).
|u .||δ .|
|0.0 .||0.005 .||0.01 .||0.02 .||0.03 .||0.04 .||0.06 .||0.08 .||0.12 .||0.16 .||0.24 .||0.32 .|
|u .||δ .|
|0.0 .||0.005 .||0.01 .||0.02 .||0.03 .||0.04 .||0.06 .||0.08 .||0.12 .||0.16 .||0.24 .||0.32 .|
2.3 Spatial Model
The spatial model is an agent-based grid model. Like the SSGA, the spatial model implements the h-iff fitness landscape. It only implements 32-bit h-iff, to reduce the computational requirements, which were considerably higher than for the SSGA. In contrast to the steady state implementation, the population was initialized with genomes in the middle local optima of the h-iff fitness landscape (0000-0000-0000-0000-1111-1111-1111-1111) with a corresponding fitness value of 64. From this starting configuration it is particularly challenging to find the global optima, since no offspring, from this starting configuration, are close to any of the global optima, which—in contrast—they can be with random initialization. Moreover, when randomizing genomes, the fitness of random individuals can by chance be so low that the population is never able to survive. Hence, the central local optimum was chosen as the genome of all individuals in the initial population.
The spatial model (Figure 6) is similar to a predator-prey model and consisted of a 250 × 250 grid where cells are either type 0 (prey) or 1 (predator). One can imagine the prey and predators to be plants and rabbits respectively, where predators were subject to evolution and each predator cell contained a binary genome. The fitness value derived from a predator's genome translates into food consumption efficiency (a metabolic efficiency). The ability to efficiently acquire food from the environment enables predators to grow faster, thereby producing more offspring. At each iteration, a given amount of biomass is added to prey cells, according to a biomass production rate that had an absolute value of +0.0016, with a maximum biomass value of 1.0. Predator cells attempted to move to a neighboring cell with 4/5 chance (moving up, down, left, or right or staying put). If the target position was occupied by another predator, or if the position was out of the grid, the predator did not move. For computational efficiency, the grid is sequentially updated from left to right and top to bottom. It was ensured that predators move only once per iteration. Predator cells reproduce with a 1/10 chance if their biomass is above twice the reproduction cost (the reproduction cost was 0.4). Offspring start with a biomass equal to the reproduction cost multiplied by 0.8 (0.32). The reproduction cost was subtracted from the biomass of the parent predator. All parameters used were chosen based on prior experimentation .
When a predator cell moves on a prey cell, it consumes the prey's biomass with an efficiency rate equal to the ratio of fitness to maximum fitness. The predator will not increase its biomass over the 1.0 limit; any unused prey biomass is left in the cell. At every iteration, predator cells lose 0.02 biomass as a maintenance cost. Predators with biomass below 0.01 are removed from the population (extrinsic mortality from starvation).
In contrast to the SSGA, the spatial model implements both intrinsic and extrinsic mortality. For intrinsic mortality we implemented a terminal age. Extrinsic mortality is an inherent property of the model that results from local competition for food between foraging agents, which is similar to the implementation of Werfel et al. . To explore the relationship between intrinsic mortality and the mutation rate, different mutation rates and terminal ages are compared. The main results indicate how often, and how quickly, the global maximum was found with 32-bit h-iff.
In both the SSGA and the spatial model, a relationship was found between the mortality rate and the mutation rate, and the performance of evolving populations with h-iff. This relationship is clear for 64-bit h-iff. However, to use 128-bit h-iff requires a lot more computational power to produce significant results. We have included some experiments using 128-bit h-iff to demonstrate that it is indeed possible using the SSGA and a mortality rate (Figure 9). The clear relationship of 64-bit h-iff SSGA and the spatial model shows how the mortality rate and the mutation rate take complementary roles for evolvability. The results furthermore depict that intrinsic mortality could lead to more evolvable populations as an alternative to a high mutation rate.
3.1 Steady State Genetic Algorithm
The number of times the global maximum was found in each of the 20 evolutionary runs is presented in Table 2. In addition, the subscript values in the table represent the average number of generations (· 103) it took the runs to find the global maximum, on average. As can be seen in the table, the relationship between the mortality rate and the mutation rate in the SSGA is very specific for finding the global maximum on 64-bit h-iff. Figure 7 shows a regression fit for this relationship between the mutation rate and the mortality rate, with an R2 factor of 0.89 for the regression fit. The results of the 20 runs using the SSGA are displayed in Figure 8 (top). To see how single runs can traverse the fitness landscape, Figure 8 (middle and bottom) depicts the fitness and diversity on the h-iff landscape over generations. Depicted by blue dots on the graphs are the individuals of a population at specific intervals.
Though the optimal ratio between the mutation rate and the mortality rate changes when varying the size of the genome, different optimal rates still exist for the larger genomes. On 128-bit h-iff, evolving populations are still able to find the global maximum within 100,000 generations when a specific ratio of mutation rate to mortality rate is used (Figure 9). With 128-bit h-iff, the maximum achievable fitness value is 448, rather than 192. Out of 20 evolutionary runs, the global maximum with h-iff, though highly unstable, was found three times in different runs when using a mutation rate of 0.03 and a mortality rate of 0.12. It was also found three times using a mutation rate of 0.06 and a mortality rate of 0.02. It was found only once when using a mutation rate of 0.02 and a mortality rate of 0.16. The other combinations of mutation and mortality rates that did not find the global maxima used similar values to those for the sweep in Table 2.
Figure 10 displays how single evolutionary runs of the SSGA and AFPO traverse the 64-bit h-iff landscape over time. As can be observed, in a single evolutionary run of 100,000 generations, the mortality rate implementation seems to exhibit more diversity while remaining at the top of the landscape. In contrast, many individuals in AFPO reside in less-fit (lower) areas of the fitness landscape, likely due to too frequent insertions of random individuals at each generation. The evolutionary progression of the SSGA implementing mortality should convey the informal principle of hill-hugging that was introduced in the previous article , since, especially compared to AFPO, the steady state implementation does not seem to occupy low-fitness regions of the fitness landscape. The two approaches were furthermore compared over 200 separate evolutionary runs that lasted 100,000 generations (Figure 11). Out of the 200 runs, only one run of AFPO found the global maximum, whereas this number was 190 out of 200 runs in the SSGA with the optimal mortality rate. AFPO performs better than SSGAs when a mortality rate below a certain threshold is implemented, but the inclusion of the optimal mortality rate made the SSGA find the global maximum more often than the AFPO.
3.2 Spatial Model
Similar to the steady state implementation, the spatial model showed a tight correlation between the mutation rate and the mortality rate for the populations that were more efficient at finding the maximum fitness with 32-bit h-iff, as seen in Table 3. Looking at the individual runs when using a terminal age of 60 (Figure 12), we observe a phenomenon similar to the SSGA, in which a low mutation rate leads to premature convergence while a high mutation rate creates unstable populations. Furthermore, if we assume that the optimal terminal age and corresponding mutation rate are defined by the number of times the global maximum is found, we can plot the relationship between the mutation rate and the terminal age for finding the optimum (Figure 13). In this case there is again a direct correlation between the mutation rate and the terminal age.
|u .||TA .|
|40 .||50 .||60 .||80 .||120 .||160 .||500 .||— .|
|u .||TA .|
|40 .||50 .||60 .||80 .||120 .||160 .||500 .||— .|
The speed of finding the global maximum in the spatial model also differs when using different mutation-rate and mortality-rate variables (Figure 14). In this case, the speed was derived from the number of cycles the spatial model ran before finding the global maximum. A different measure of speed would be to count the number of individuals that had been simulated before the maximum had been found. However, the only real difference in Figure 14 when comparing mutation rate 0.02 and terminal age 50 with mutation rate 0.16 and terminal age 500 is that the latter needs to simulate significantly fewer individuals before the maximum is found (two-sided Mann-Whitney u-test; p-value 0.008). The difference in speed in terms of the number of cycles was not significant in this comparison (two-sided Mann-Whitney u-test; p-value 0.2); the median number of individuals simulated before finding the global maximum was 971,792 in the low-terminal-age scenario and 188,148 in the high-terminal-age scenario. Thus, a higher terminal age in this comparison is needed to simulate fewer individuals. The speed plot using the number of individuals as a measure, instead of using the number of cycles, looked almost identical to Figure 14 and is therefore not included in the article. For evolutionary algorithms in general, the number of individuals simulated should be minimized, though producing more individuals within a short time might be more valuable for real-world populations.
Another interesting result when not using a terminal age is that although the average age of the individuals in the population remains relatively fixed across different mutation rates, the maximum age is significantly higher in high-mutation-rate scenarios (two-sided Mann-Whitney u-test; p-value 7 × 10−8). The difference between the maximum age in the low-mutation-rate scenario (mutation rate 0.03) and in the high-mutation-rate scenario (mutation rate 0.24) is displayed in Figure 15. These results were interpreted as the fittest individuals being unable to produce many functional offspring due to the high mutation rates. This meant that the older, fitter individuals had a higher chance to outcompete the other, likely less fit individuals in the populations. The elite thus became much older in scenarios where there was a high mutation rate—when looking at no terminal age, the maximum age of the population under high mutation rates could grow as high as 20,000 cycles.
As displayed in Table 3, as high mutation rates also lead to a greater number of less-fit individuals in the population, the mutation rate is necessarily low; otherwise the population would go extinct, as denoted by ϵ in Table 3. Both a mortality rate and a mutation rate could thus lead to an error catastrophe, where a species goes extinct due to excessive mutations. In conclusion, the results indicate that the mortality rate and the mutation rate shape the evolvability of the population simulated in the spatial model.
Our results show that evolvability is greatly influenced by the mutation- and mortality-rate ratios in both evolutionary algorithms and a spatial model. In particular, the h-iff function, despite its deceptiveness, can be traversed by an SSGA through simply including an indiscriminate mortality rate. This allows it to find the global optima with 64-bit h-iff, as well as with 128-bit h-iff. SSGAs with no chance of removing elite individuals are prone to premature convergence, as demonstrated by the few times the global maximum was found when no mortality rate was implemented (e.g., Figure 8(a)). Likewise, random search is approached when both the mortality rate and the mutation rate are too high. As in the SSGA, the intrinsic mortality rate also influences the rate at which the global maximum with h-iff is found in the spatial model. Therefore, evolvability is also enhanced in models where mortality already arises from external mortality factors as a result of local competition. Since a fitness landscape in nature is likely to be highly convoluted—and possibly deceptive—we speculate that programmed aging could be, as Goldsmith  has mentioned, beneficial for the evolvability of a population. The better a species can traverse the fitness landscape without going through low-fitness regions, the more plausible it is that the population will find more adaptive traits. This, in turn, makes the population better and potentially increases its ability to cope with changing environments.
It is relevant to note that the mortality factor we are considering in this article is very easy to implement—simply remove individuals from the population at random to increase the evolvability of the population. Lehman  used extinction events to increase the evolvability of a population, and it has been shown that the mortality rate and the extinction events both lead to an algorithm that can have increased evolvabilty . As long as some sort of mortality mechanism is implemented, it is likely that the population's evolvability is increased for deceptive fitness landscapes.
A common problem with EAs is that optimal hyperparameters highly depend on the given domain. EAs that incorporate generational replacement  inherently implement a mortality mechanism, since the entire population is replaced by a new population of offspring every generation (when no elitism is implemented). Moreover, deletion in a steady-state genetic algorithm has also been investigated (e.g., for dynamic environments) and shown to perform similarly to generational EAs . The application of a mortality rate in EAs can therefore also elucidate whether such mechanisms should be implemented in an existing EA to better traverse the fitness landscape.
AFPO, developed to combat premature convergence, relies on a new random individual being inserted in a specific region close the maximum fitness of the landscape by accident . From this starting point, the new individuals on the Pareto front in this age category need to quickly find a good solution before they are outcompeted by a younger strain climbing another local hill. A potential improvement to AFPO is thus also to insert a new random individual at intervals, not every generation, so the individuals in the new age category have time to climb a local hill, preventing them from being out competed by chance. In the SSGA, despite the occasional loss of the best individual in a population, the entire population of individuals remains close to the top of the fitness landscape. Figure 13 shows that the optimal mutation rate saturates as the terminal age increases. The increment in mutation rate as the terminal age increases over 1000 (up to infinity, or no terminal age) is negligible. As described in the mutation accumulation theory by Medawar , this terminal age is so high that selection would not be influenced by it, since the average individual probably dies before reaching this age. The 64-bit SSGA simulations frequently lost an optimal solution previously found (Figure 8(b)). This happens less frequently in the spatial model (Figure 12), which may result from increased robustness of the spatial model as well as from the genome size (which is smaller in the spatial model). In addition, the spatial model has a higher probability of keeping the best solution in the population, since a fitter population can sustain more individuals than a less fit population. Its metabolic efficiency is, on average, higher. This increased stability ensures that the global maximum is not lost in the spatial model.
There are a few ways in which we see how an intrinsic mortality rate could arise in natural populations. A continuous rate of evolvability can enable one population to outcompete other, less evolvable populations. In addition, individuals in a population with an intrinsic mortality rate might keep their optimal mortality rate steady when external factors fluctuate. Finally, a higher mortality rate and lower mutation rate might prevent scenarios such as error catastrophes commonly seen in nature.
The mutation rate and the mortality rate have an optimal ratio that depends on multiple factors, such as reproduction and development, crossover, and selection pressure. However, we think that the few essential factors that always shape evolvability are mortality, reproduction, and variation. These factors can be altered through different mechanisms:
Variation. Factors like crossover and mutation rates influence the difference of the location in the search space of the progeny from that of their parents.
Reproduction. The population size and density, foraging efficiency, reproduction rate, and so on might shape how many new individuals are born.
Mortality. Various factors, extrinsic or intrinsic, dictate the number of deaths and thereby the available space for offspring.
In addition, species in natural environments suffer from both intrinsic (aging) and extrinsic (predation, accidents, and parasitism) mortality. Extrinsic mortality is known to fluctuate, both in predictable ways (seasons) and depending on external factors (diseases, variable predator pressure). As there is a clear relationship between the mortality and the mutation rate for optimal evolvability, this means that such fluctuations in mortality rates could affect the evolvability of populations. Evolving an intrinsic mortality factor may alleviate the problem of needing a fluctuating mutation rate. When an external mortality rate is high, aging is not a dominant factor. If external mortality is decreased, then intrinsic mortality prevents the death-rate-to-mutation-rate equilibrium from being out of balance, potentially preserving evolvability in a population. A possible benefit to senescence could thus be to keep the evolvability of a population steady under conditions of fluctuating extrinsic mortality rates. As discussed by Herrera et al. , intrinsic mortality seems to be beneficial in changing environments, and therefore a steady optimal evolvability rate might be required.
An evolving population might benefit from low mutation rates and high mortality rates, since it gradually changes the genetic information in the population's progeny instead of producing drastic changes. In a population with a low mutation rate and a high mortality rate, the next generation of individuals would be genetically closer to the previous generation, but can be far away in the long run. A high-mutation-rate and low-mortality-rate implementation instead creates a next generation that consists of individuals that are already more distant from their parents than in the previous implementation. It is thus possible that a higher mortality rate prevents the occurrence of error catastrophes that can lead to the extinction of a species.
The results of this article, in conjunction with the work shown by [16, 26], suggest that mortality, or aging, is beneficial to the evolvability of a population, and could therefore be programmed. In addition, we have argued how the increased evolvability leads to a selective pressure that might result in an increase of the rate of mortality. As mentioned in the introduction, mortality reduces the personal fitness of an individual drastically, though the inclusive fitness might be increased. Therefore, the death of an individual can altruistically bolster the evolvability of an evolving population.
An explicit relationship between the mutation rate and the mortality rate for optimal evolvability on a deceptive fitness landscape in both spatial and non-spatial evolutionary models has been presented. Not only do these results increase our understanding of senescence, they also hold potential benefits for applications to evolutionary algorithms and robotics. As an alternative to proposed theories of aging showing how intrinsic mortality is advantageous for altruistic aging, we claim that intrinsic mortality governs evolvability and that it is thereby a potentially favorable trait, ultimately supporting the theories of programmed death. Moreover, in scenarios of fluctuating extrinsic mortality rates, an intrinsic mortality rate would keep the evolvability stable—a potentially relevant evolutionary benefit of intrinsic mortality. Our model predicts an optimal relationship between mutation rates and mortality. However, attaining peak evolvability through large increases in mutation rates risks catastrophic errors that could lead to the extinction of the species. Since the experiments presented in this article were done using a deceptive fitness landscape, it remains speculative whether nature contains deceptive dimensions that are traversable through mortality-induced evolvability.
This project has received funding from Flora Robotica, a European Union Horizon 2020 research and innovation program under the FET grant agreement, no. 640959. Computation and simulation for this article were supported by the Vermont Advanced Computer Core, University of Vermont.
See chapter 2 of  for a discussion on evolvability.
The population or gene pool can also be viewed as a vehicle that can be defined as “any unit … which houses a collection of replicators” [7, p. 173].
The source code for the SSGA and the agent-based grid model can be found here: https://github.com/FrankVeenstra/ALife2018.
Some preliminary results on how the experiments translate to other functions can be found in chapter 3 of .