This article presents a lightweight platform for evolving two-dimensional artificial creatures. The aim of providing such a platform is to reduce the barrier to entry for researchers interested in evolving creatures for artificial life experiments. In effect the novel platform, which is inspired by the Sodarace construction set, makes it easy to set up creative scenarios that test the abilities of Sodarace-like creatures made of masses and springs. In this way it allows the researcher to focus on evolutionary algorithms and dynamics. The new indirectly encoded Sodarace (IESoR) system introduced in this article extends the original Sodarace by enabling the evolution of significantly more complex and regular creature morphologies. These morphologies are themselves encoded by compositional pattern-producing networks (CPPNs), an indirect encoding previously shown effective at encoding regularities and symmetries in structure. The capability of this lightweight system to facilitate research in artificial life is then demonstrated through both walking and jumping domains, in which IESoR discovers a wide breadth of strategies through novelty search with local competition.
Research in artificial life often depends on evolving artificial creatures. For example, such creatures are essential in experiments in open-ended evolution [3, 4, 14, 19, 22, 23, 30], morphological innovation [2, 9, 10], and locomotion [5, 12, 17]. While the results of such experiments are often compelling, their computational cost and complexity present barriers to entry to the field for new researchers.
For example, researchers investigating open-ended dynamics cannot focus exclusively on the most effective incentives for open-endedness, because the experiment cannot even get off the ground without a creature or animat upon which various hypotheses can be tested. Although some researchers naturally will prefer to build the entire domain themselves, in some cases a convenient and lightweight solution could allow more to enter the field by focusing on bigger questions with a smaller initial implementation burden.
This article, which is an invited extension of an earlier one in the 12th European Conference on Artificial Life (ECAL-2013), addresses this need by introducing such a lightweight creature-evolution domain. The new domain is inspired by a previously existing project called Sodarace [15, 16] in which two-dimensional simulated creatures made of masses, springs, and muscles ambulate according to their particular morphological configuration. In this initial realization of Sodarace, humans designed the body morphologies by hand and then raced them competitively. The result was a broad collection of diverse human-designed morphologies (Figure 1a) that resemble to some extent the output one might expect from an artificial life world. Later versions of Sodarace added an evolutionary module that could in fact evolve new morphologies without human assistance, but because the evolutionary component is relatively simple, these evolved morphologies capture only a small subset (Figure 1b) of the diverse possibilities suggested by the many known human designs.
The problem is that evolving more interesting creatures for Sodarace would require a sophisticated encoding that is capable of generating diverse yet regular structures that can ambulate effectively. This article enables such evolution by introducing an indirect encoding for Sodaracers based on compositional pattern-producing networks (CPPNs) . One reason CPPNs make sense for encoding Sodarace-like creatures is that their springs and muscles exist along connections, which means that their morphologies are in effect connected graphs. CPPNs in turn are commonly applied to evolving the connectivity of graphs within the HyperNEAT algorithm [7, 25], which is usually applied to evolving the connectivity of artificial neural networks. Instead, in this work, called indirectly encoded SodaRace (IESoR), the CPPNs encode the morphological connectivity of the Sodaracers, thereby transferring the CPPN's ability to encode regularities and symmetries within networks to a new domain.
The main benefit of IESoR then is that it provides an established indirect encoding for evolving in a popular, lightweight creature domain. The potential of this system is demonstrated in this article by evolving both walkers (similarly to the intent in the original Sodarace) and jumpers. Variety is achieved in both tasks through the novelty search with local competition  approach, which is designed to collect a diversity of solutions within the same domain. The resultant variety demonstrates that IESoR is indeed a sufficiently rich domain that it could in principle form the basis of further experiments in artificial life, thereby opening the field to more researchers and offering a platform with clear visual outcomes.
The next section summarizes the history of Sodarace and reviews Hyper-NEAT. Section 3 then details the IESoR approach and its CPPN-based encoding. Next, Section 4 explains the experimental design, and Section 5 provides the results. Overall implications are discussed in Section 6, followed by a brief Conclusion in Section 7.
This section contains an overview of Sodarace and MINS, the online projects that serve as the inspiration for IESoR, as well as a brief review of the NEAT and HyperNEAT methods used to encode creature morphology in IESoR.
The Sodarace project is a simple two-dimensional physics world consisting entirely of masses, springs, and basic oscillatory muscles [15, 16]. The goal in Sodarace is to create virtual robots and race them in different environments. Both the robots and the environments are usually handcrafted by users. However, to aid in creating robots, a construction kit is provided to allow discovery and exploration by the community [15, 16].
The Sodarace project was originally conceived as a type of online Olympics meant to test humans against machine intelligence at the task of designing robot racers. In fact, one redesign of the software includes an evolutionary algorithm that optimizes morphologies for racing. Reflecting the software's educational aspirations, an online repository of creatures and all relevant software packages is accessible in a centralized location . At the peak of popularity, Sodaconstructor, the tool for creating the creatures, was played by about a million active users , suggesting its potential as a platform for exploration and discovery.
Though Sodarace was a beacon for user creativity, the project itself was created over a decade ago, and the community has declined since then. Nevertheless, the peak popularity of the project suggests the domain has wide appeal, though some aspects of the original software make Sodarace inaccessible to academic research. For example, because it is a closed source Java applet, certain parameters of the races cannot be modified because the Sodaconstructor user interface does not provide user access.
To address the obstacle to research, Stefan Westen created MINS Is Not Sodarace (MINS), an open source replica of the Sodarace environment . MINS is fully compatible with Sodarace, allowing the user to import design and environments from Sodaconstructor. By creating an alternative open source environment, MINS allows adjusting parameters that are hardcoded inside the Sodarace domain. In an effort to curb cheating by Sodarace creatures, MINS alters the environment ceiling size, starting velocities, and maximum movement speeds.
MINS is an inspiration for the work in this article, in part because the variety of creature types found with the Sodaconstructor suggests the space of creatures is rich. MINS also shows that replicating the Sodarace environment is feasible and lightweight, while maintaining backward compatibility. The primary principle extracted from Sodarace and MINS is the use of masses, springs, and muscles to construct varied creatures inside a customizable physics environment .
2.3 NEAT and HyperNEAT
Evolving morphology and control is familiar in artificial life [1, 2, 8, 10, 13, 21]. In this spirit, recent additions to Sodarace include a utility with an evolutionary algorithm for creating and optimizing racers inside the software . In particular, inside the Sodarace Kiosk, users are presented with a limited interface for designing a creature. The user can adjust the number of virtual nodes and muscles along with the amplitude and frequency at which the muscles oscillate. The evolutionary algorithm then searches, constrained by those parameters, through selection and mutation to find the fastest racer possible. Inside the Sodarace Kiosk, by default creatures appear to be circular in form with crisscrossed inner connections. With gravity, the resulting creatures take on the shape of a semicircular blob (Figure 1b). The Sodarace community refers to this creature morphology as being amoeba-like. There is a Sodarace utility called the Amoebamatic that aids users in constructing these amoeba racers.
However, the most interesting handcrafted creatures from Sodarace generally do not exhibit amoeba-like properties. That is, the included genetic encoding is highly constrained to a small subset of all the interesting possibilities. Ideally, the encoding and evolutionary algorithm for evolving such racers would be able to search a wide breadth of possible creatures, which would make this kind of domain relevant to artificial life. Yet to efficiently search such a space requires a principled encoding capable of searching variable levels of complexity.
The first step towards this end in IESoR is the NeuroEvolution of Augmenting Topologies (NEAT) algorithm [26, 27]. Although NEAT was originally introduced as a method for evolving artificial neural networks (ANNs), a major appeal of NEAT is its ability to evolve increasingly complex structures of any type, so that evolutionary search is not limited to a fixed space of possibilities.
Conveniently for IESoR, a method called Hypercube-based NEAT (HyperNEAT) [7, 25] builds on NEAT to help it encode large connectivity patterns with natural regularities like symmetry and repetition of structure. While such regularities are useful for neural networks, they can also in principle benefit bodies made of connections and joints in a similar way.
The key ingredient behind HyperNEAT is an indirect encoding called a compositional pattern-producing network (CPPN) . The idea behind CPPNs is that geometric patterns can be encoded by a composition of functions that are chosen to represent common regularities. The internal structure of a CPPN is a weighted network, similar to an ANN, that specifies which functions are composed and in what order, which means that instead of evolving ANNs as it normally does, NEAT can evolve CPPNs that generate a connectivity pattern across a network.
The difference in this article is that the pattern encoded by CPPNs is interpreted as a body plan for a Sodarace-like creature instead of a neural network. In fact, in a significantly different domain, Auerbach and Bongard  encoded the bodies of three-dimensional ambulating creatures with CPPNs. The indirect CPPN encoding can compactly represent patterns with regularities such as symmetry, repetition, and repetition with variation [20, 24], which are also exhibited by many natural organism morphologies on Earth. In fact, part of the inspiration for CPPNs derives from observations of natural bodies .
To understand how a composition of functions could represent these regularities, we note that simply by including a Gaussian function, which is symmetric, the output pattern of a CPPN can become symmetric. A periodic function such as the sine creates segmentation through repetition. Most importantly, repetition with variation (e.g., as with the fingers of the human hand) is easily discovered by combining regular coordinate frames (e.g., sine and Gaussian) with irregular ones (e.g., the asymmetric x-axis). For example, a function that takes as input the sum of a symmetric function and an asymmetric function outputs a pattern with imperfect symmetry.
In this way, CPPNs produce regular patterns with subtle variations. The potential for CPPNs to represent patterns with motifs reminiscent of patterns in natural organisms has been demonstrated in several studies [20, 24] and suggests that such an encoding has potential in the domain of two-dimensional creatures.
Formally, CPPNs in HyperNEAT are functions of geometry (i.e., locations in space) that output connectivity patterns whose nodes are situated in n dimensions, where n is the number of dimensions in a Cartesian space. Consider a CPPN that takes four inputs labeled x1, y1, x2, and y2. This point in four-dimensional space also denotes the connection between the two-dimensional points (x1, y1) and (x2, y2), and the output of the CPPN for that input thereby represents the weight of that connection (Figure 2). By querying every possible connection among a pre-chosen set of points in this manner, a CPPN can produce a connectivity pattern, wherein each queried point is a node position. Because the connections are produced by a function of their endpoints, the final structure is produced with knowledge of its geometry. In effect, the CPPN paints a pattern on the inside of a four-dimensional hypercube that is interpreted as the isomorphic connectivity pattern; this is the origin of the name Hypercube-based NEAT (HyperNEAT). Connectivity patterns produced by a CPPN in this way are called substrates, so that they can be verbally distinguished from the CPPN itself, which has its own topology. While the substrate in the original HyperNEAT is interpreted as an ANN, in IESoR the substrate is a creature's body, as explained next.
This section describes the implementation details of IESoR, and explains the variant of HyperNEAT that enables it to create and evolve body plans.
IESoR implements three primary properties derived from Sodarace and MINS (Figure 2):
The environment is two-dimensional, and creatures consist solely of masses, springs, physical joints, and muscles.
In creature bodies, masses are implemented by nodes, and springs are connections attached at the joints.
Muscles manipulate the length of connections, leading to motion.
3.2 Encoding Morphologies with HyperNEAT
Bodies inside IESoR consist of masses with variable or fixed length constraints. Each constraint, or connection, is represented by a distance joint in Box2D (i.e., a constraint on the length between two masses) and has three distinct properties:
The joint is of either variable or fixed length (i.e., a muscle or a bone).
The change in distance during muscle contraction is the muscle amplitude.
The phase shift of the sinusoidal function controlling muscle length is the muscle phase.
Fixed-length connections, or bones, do not receive a magnitude or phase from the CPPN.
Recall that HyperNEAT paints a four-dimensional pattern across the weights of a network by querying a CPPN for every pair of nodes in the substrate. The insight in this article is to take this concept of a substrate and extend it to two-dimensional morphologies. Instead of painting a pattern of weights across the substrate, the CPPN encodes both what joint constraints should exist between masses on a two-dimensional plane and their three virtual properties (i.e., bone or muscle, amplitude, and phase). For this purpose, the CPPN requires four outputs (as shown in Figure 2).
Before clarifying how a HyperNEAT substrate can be used to represent a morphology, it is important to consider the placement of bones and muscles in natural body plans. The skeletal system is crucial to mobility at a fundamental level. Equally important to where bones are placed in a body plan is the information of where bones are not placed. If a rough representation of the human body is to be drawn on a small grid of dots, the principle of symmetry is as important as the fact that there is no bone connecting the tip of the foot to the top of the skull. Morphologies generated in IESoR ideally also should usually respect this simple principle of locality.
Conveniently for this purpose, HyperNEAT can be expanded with a special link expression output (LEO)  to generate an expression pattern that controls whether connections are expressed at different locations independently of other CPPN outputs. In Verbancsics and Stanley , HyperNEAT with LEO was seeded with a bias towards favoring locality, but evolution could adjust this bias during search.
To generate a morphology using an n-by-n grid of nodes as the substrate, for each node location in the substrate (Figure 2), the CPPN queries all other node positions. The (x, y) coordinates of nodes i and j are denoted as (xi, yi) and (xj, yj), respectively. The input into the CPPN is thus xi, yi, xj, yj, and there are four outputs. First, the LEO (which is a step function) is checked for a positive value. If the LEO is positive, a connection is placed between nodes i and j from (xi, yi) to (xj, yj). Then the output that determines whether the connection is a bone or a muscle is queried. If the output value is below a predefined muscle threshold, the connection becomes a fixed-length constraint. Otherwise, the constraint is a muscle, and the amplitude and phase of the muscle contraction are read from the remaining two CPPN outputs. Finally, to further reduce complexity in the resultant morphologies and keep computational costs low, pairs of points at greater distance than a third of the diagonal length of the substrate are not queried while constructing the two-dimensional creatures. An example of a fully constructed morphology is shown in the lower right of Figure 2.
After assembling the masses and joints, the bodies are placed in a simple Box2D environment consisting of the ground, gravity, and friction. As the world is simulated, muscles oscillate according to the amplitude and phase values set by the CPPN, while bones preserve a fixed length. Creatures occupy distinct Box2D environments, and nodes cannot collide with each other.
Though its creatures are mainly handcrafted, Sodarace shows that the space of possible two-dimensional body types is likely filled with creatures capable of movement. As noted in the Background section, the Sodarace Kiosk went on to create an automated approach to generating creatures, but resulted in a highly restricted space of bodies. In contrast, two experiments described in this section are constructed with IESoR to find Sodaracers capable of both walking and jumping behavior. These experiments are designed to show that not only is an automated approach capable of designing two-dimensional walkers and jumpers, but the method can also produce a wide variety of different means of locomotion, thereby giving hope for further application of Sodarace-like creatures in artificial life.
4.1 Novelty Search and Local Competition
To best demonstrate the morphological diversity possible in IESoR, Pareto multi-objective search (based on NSGA II) , including both novelty and local competition , is implemented to explore the space of body types. Lehman and Stanley  first applied Pareto multi-objective search with novelty and local competition to yield a diverse group of ambulating three-dimensional morphologies, all within a single run of evolution. Maintaining and exploiting diversity across evolution is both an impressive and an important part of validating the potential for future artificial life research with IESoR.
The first of the three objectives that make up novelty search plus local competition is novelty search, which was introduced by Lehman and Stanley [11, 12] to avoid the common pitfall of evolution prematurely converging on a deceptive objective. Novelty search aligns well with the aims of these experiments because the hope is to find a diversity of novel creatures. Joachimczak and Wrobel  have shown already that novelty search can be effective for this purpose. The characterization of creature novelty for the novelty search component can significantly affect evolution and strongly bias the resulting creatures discovered. In these experiments, novelty search characterizes creatures by their width, height, and mass (as measured by the number of nodes and the sum of the connection lengths) at the first time step of the simulation, which should lead to a visually diverse population. The novelty metric is the squared Euclidean distance separating two individuals in this characterization space, and thus the novelty of a creature is proportional to how different its starting morphology is from that of other creatures currently in the population. Such a characterization space especially encourages creatures with varying widths, heights, and masses.
The second objective, local competition, forces individuals to compete only with those who are characterized as similar . The idea is that within novelty search it is possible to push individuals who are similar with respect to the behavior characterization to compete locally to be the best of their type. That way, globally novelty search probes a wide variety of possibilities, but locally individuals optimize to be the best they can. In IESoR, creatures who are locally close share similar widths, heights, and masses, ideally indicating a similar morphology. Local competition is the mechanism for pressuring individuals with related morphologies towards more effective locomotion.
Importantly, by changing the local competition objective, the search can be heavily biased to find morphologies capable of fundamentally different movement strategies. Therefore, the local competition component is the primary source of difference between the two experiments described in this article. For the first experiment, the local competition objective is the horizontal distance traveled by an individual. The second experiment measures instead the maximum vertical distance above the physical ground achieved during the individual's simulation. That is, the first experiment encourages walking, while the second experiment encourages jumping. Note that by modifying only this second objective, these experiments allow the direct comparison between morphologies generated by both experiments to best isolate the effects of local competition during search.
As in , the Pareto multi-objective search has three objectives: novelty, local fitness, and finally genotypic diversity. The genotypic diversity objective encourages exploring innovative genotypes by assigning higher values to more novel genotypes. That way, new genotypes created by HyperNEAT are not initially penalized and thereby have a chance to optimize to reach their potential. This genetic diversity objective is in effect a multi-objective-compatible substitute for the usual speciation mechanism in NEAT, which serves the same purpose. Additionally, the genotypic diversity objective is also localized within the characterization space; similar in motivation to local competition, local genetic diversity ensures that genotypic diversity is not only exploited in those characterization niches in which such diversity is incidentally most easily expressed.
In all setups, the distribution of individuals in behavioral space as well as their overall performance is recorded. The idea is to quantify how much morphological diversity is discovered and maintained and how well each behavioral niche is being exploited overall throughout a run.
4.2 Experimental Parameters
The overarching multi-objective algorithm is based on NSGA II . The population size is 120 for all runs, and the walking experiment ran for 1,200 generations, whereas the jumping experiment ran for 300 generations. The nearest-neighbor size for novelty search and local competition is 20. The three morphology dimensions used to characterize novelty (width, height, and mass) are rescaled so that their values fill the range between zero and three. The selection method for NSGA II was tournament selection (with tournament size two), and other parameters followed precedent , which in turn used the parameters of .
The intent of these experiments is to demonstrate that a wide variety of walkers exists in the encoding space defined by IESoR, thereby establishing the viability of IESoR for future ALife research. Thus, as opposed to machine learning experiments aimed at demonstrating optimality, the aim in these experiments is to show both diversity and competence. Recall also that novelty search plus local competition is designed to return a significant coverage of possible solutions from a single long run. There is precedent for demonstrating the diversity that results from such a search. For example, Lehman and Stanley  measured the height and mass of three-dimensional morphologies from novelty search plus local competition to show the breadth of morphologies discovered by evolution, and Joachimczak and Wrobel  used principal component analysis (PCA) to demonstrate coverage across morphological space after novelty search. Following this precedent, to quantify IESoR's ability to create diverse walkers, PCA is run across characterizations in both experiments of all generated creatures during evolution to create a visualization of the resultant diversity.
In particular, to characterize morphological diversity in IESoR for the purpose of visualization, three dimensions that describe gross creature characteristics (viz., width, height, and mass) are projected into a two-dimensional space by the PCA algorithm. However, while PCA with this information can reveal the diversity across the morphological space, the goal of this analysis is to give a sense of the competence of such creatures as well. That way it becomes possible to observe the diversity of competent creatures instead of just diversity overall. Therefore, in the visualization of the PCA output in both Figures 3 and 5, to ensure that each graph shows the diversity of only competent walkers or jumpers, only points for walkers that ambulate beyond 200 units and jumpers that launch higher than 10 units off the ground are displayed. Furthermore, each point's radius is proportional to the absolute performance within the task, either walking or jumping.
Because thousands of points result, the visualization is further refined to reduce clutter and ensure that each point represents a genuinely unique individual. For this purpose, the plane is discretized into 40 × 40 equal-sized bins. Creatures are placed into bins according to the coordinate assigned by the PCA process. Conceptually, each bin thus represents a similar area in morphological space, and the creature assigned to a bin that performed the best among all in that bin is chosen as the representative of that bin. That way, each circle in Figures 3 and 5 shows the best performance for the morphological class represented by its bin, and each circle represents a distinct morphological class. Any bin without a representative is shown as empty space in both figures.
For the ambulation experiment, of the 1,600 possible bins, 450 are filled with individuals who can ambulate the minimum distance, covering in total 28.1% of all possible bins. Furthermore, the visualization in Figure 3 exhibits the breadth of coverage of competent morphologies. In effect, IESoR with novelty search plus local competition uncovered hundreds of unique and effective ambulation methods covering a significant breadth of conceivable strategies.
Additionally, for the jumping experiment, of the 1,600 possible bins, 500 are filled with individuals who launch off the ground to a minimum height of 10 units, covering 31.25% of all possible bins. Figure 5 depicts a range of morphologies capable of making at least one jump above 10 units.
Just as important as this quantitative perspective is a qualitative analysis of the breadth of behaviors. It is important to note that every behavior in Figures 3 and 5 can be viewed at http://eplex.cs.ucf.edu/iesor/live/walk/ and http://eplex.cs.ucf.edu/iesor/live/jump/, respectively, through a special online interface where the user can click on any point and see the corresponding creature behavior. This fast interactive visualization of hundreds of creatures is possible in part due to the lightweight, inexpensive nature of Sodarace-like creatures, which is one of their potential advantages for researchers in artificial life. Figures 3 and 5 both show a sampling of morphologies, while Figures 4 and 6 show a subset of those at different stages of motion.
An additional important further qualitative observation is the significantly broader diversity seen in IESoR than in the original Sodarace evolver's amoeba-like creatures shown in Figure 1b. Among those that can be observed are gaits based on loping (degrading into pushing) (Figures 3a and 4a), pogo-stick hopping (3b and 4b), multiple cascading octopus legs (3c and 4c), dragging (3d and 4d), complex galloping (3e), sliding and pumping (3f), and bouncing into a long dive (3g). Some strategies depend on an initial burst of propulsion, while others rely on stable and consistent ambulation. Some of the very best gaits (largest circles in Figure 3) involve galloping or hopping, though even among the very best the diversity of approaches is significant.
Note that by setting the local fitness objective to be the jump height, IESoR demonstrates a capability to find Sodaracers with behaviors not originally anticipated by the Sodaconstructor. As the name implies, Sodaracers are designed for racing, but IESoR is able to discover a breadth and variety of jumping Sodaracers just as effectively as walking Sodaracers. As with walking Sodaracers evolved by IESoR, creatures found during the jumping search employ a variety of behaviors to excel at the local fitness objective. Some jumping behaviors rely heavily on symmetric bodies and symmetric muscle contractions (Figures 5a and 6a) to launch into the air, while other creatures engage multiple muscles at a single node to spring upwards (5c and 6c). As is the case with creatures from Figure 5b and d, even if two individuals have nearly identical initial morphologies and are therefore at the same location in the PCA space, the method for propelling the body into the air can be entirely distinct (6b and d).
The main result is that Sodarace-like domains do contain a diversity of viable walkers and jumpers that can be systematically discovered with the right encoding and selective pressures. Unlike with the early evolver application built for Sodarace, morphologies evolved in IESoR do not exhibit only a single stereotypical organizational motif. Instead, they ambulate or jump in many different ways, from legged-style locomotion to serpentine pulsation to symmetric springlike hopping to curling in the air for more height. Interestingly, to discover creatures capable of vastly different types of movement, from walking to jumping, only the local competition objective needs to be altered, suggesting the potential for more elaborate applications of this kind of technology in the future.
Investigations relevant to artificial life on open-ended evolution [4, 14, 19, 23, 30] and diversity maintenance [13, 18] can thus be quickly set up and conducted in the future with IESoR. Creatures can also potentially move beyond forward ambulation to more complex interactions such as foraging or predation. To facilitate such future applications, code for IESoR is available at https://github.com/OptimusLime/IESoR.
By demonstrating that creatures evolved by IESoR can both ambulate and jump, the experiments in this article suggest that it can serve as a viable lightweight platform for future ALife experiments. Furthermore, not only were the solutions evolved for walking and jumping effective, but they were also diverse, supporting the insight that the space of Sodarace-like creatures is large enough to support genuinely creative processes of exploration and discovery. The implication is that the up-front effort required by many ALife experiments can potentially be avoided if desired by adopting this domain to make it easier to focus on higher-level research questions.
This work was supported by the National Science Foundation under grant IIS-1002507. Any opinion, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation. Special thanks to Peter W. McOwan and Edward J. Burton for creating the original Sodarace, and to Stefan Westen for creating MINS.