Abstract

We investigate a hierarchical approach to robot control inspired by joint-level control in animals. The method combines a high-level controller, consisting of an artificial neural network (ANN), with joint-level controllers based on digital muscles. In the digital muscle model (DMM), morphological and control aspects of joints evolve concurrently, emulating the musculoskeletal system of natural organisms. We introduce and compare different approaches for connecting outputs of the ANN to DMM-based joints. We also compare the performance of evolved animats with ANN-DMM controllers with those governed by only high-level (ANN-only) and low-level (DMM-only) controllers. These results show that DMM-based systems outperform their ANN-only counterparts while also exhibiting less complex ANNs in terms of the number of connections and neurons. The main contribution of this work is to explore the evolution of artificial systems where, as in natural organisms, some aspects of control are realized at the joint level.

1 Introduction

Animals exhibit remarkable feats of agility and locomotion. Underlying and supporting these abilities are complex, intertwined neuromuscular and musculoskeletal systems that evolved concurrently [24, 58]. Joints are controlled by signals from the brain propagating through the somatic nervous system to groups of muscles, whose collective responses produce movement. Importantly, the behavior of a joint also depends on the composition and orientation of the constituent bones, tendons, and muscles [1, 2].

In robots, joint control operates very differently. Typically, robotic joints are based on motor-driven actuators that receive commands directly from a high-level controller, such as an artificial neural network (ANN), which is also responsible for governing aspects of overall behavior. Although improvements in robotic locomotion have been demonstrated through the coevolution of controllers and morphological attributes [27, 28, 45], most robotic systems do not exhibit the agility and dexterity seen in natural organisms.

In this article, we investigate a hierarchical approach to robot control inspired by joint-level control in animals. The method integrates a high-level controller, consisting of an ANN, with joint-level controllers called digital muscles [37]. Rather than simulating the behavior or structure of physical muscles, digital muscles abstract the relationships between control signals and morphological structures at the joint level. Specifically, the orientation of muscle nodes around the joint, along with their responses to high-level signals, evolve concurrently with each other and with the ANN.

An earlier study [37] demonstrated that the digital muscle model (DMM) can produce effective gaits in 3D animats, even when using only a simple sinusoid as the high-level control signal. We later explored pairing a high-level ANN controller with DMM-based joints in evolving ball balancing and gaits for legged animats [38]. There, we found that the connection strategy between the ANN and the DMM joints affected overall performance and influenced the complexity of the evolved ANN. In this article we further explore the coupling of ANN controllers with DMM joints, focusing on a single platform, a quadrupedal animat, and the relationship between performance and the complexity of the evolved ANNs. We examine both the characteristics of evolved populations and the specific traits of the most effective individuals. As in our earlier study, we consider two different ways of connecting ANN outputs to DMM joints. These hierarchical ANN-DMM controllers are compared with each other, as well as with systems governed by only high-level (ANN-only) and only low-level (DMM-only) controllers. For different controller configurations, we measure both animat performance, in terms of distance traveled in a specified time, and the ANN complexity, in terms of the number of connections and neurons. To determine whether the complexity of the ANN can be reduced without sacrificing performance, we conduct a series of treatments using phased searching [25], in which ANN evolution alternates between complexification and simplification of the ANN.

The main contribution of this work is to report results on the evolution of artificial systems where, as in natural organisms, some aspects of control are realized at the joint level. The results of evolutionary experiments demonstrate that, when combined with high-level ANNs, ANN-DMM controllers outperform their ANN-only counterparts while also exhibiting less structural complexity in the ANN component. Furthermore, the phased searching experiments demonstrate that the ANN complexity can be further reduced with only a slight decrease in animat performance. Collectively, these results indicate that for ANN-DMM controllers, the low-level aspects of control can be compartmentalized or offloaded from the ANN, potentially freeing it to focus on other tasks.

2 Background and Related Work

Evolutionary robotics [5, 11, 20, 32, 40, 49] borrows concepts from natural evolution and applies them to the design of simulated or physical robots. Many studies in this area focus on optimizing controllers for robots with fixed morphologies [12, 13]. Controllers such as ANNs [59] and central pattern generators (CPGs) [34] are amenable to evolutionary optimization and have produced various forms of robotic locomotion, including salamander gaits [27], bipedal walking [45], and crawling [28]. Typically, these controllers generate outputs governing the movement of each joint, such as the desired angles of actuators, in addition to handling high-level decision making. In natural organisms, however, intrinsic properties of muscles themselves contribute to both stability and function [23]. For example, the tendon network of the human hand has been shown to perform active tension modulation, independent of the neural system [55].

Such observations have led researchers to explore the concurrent evolution of morphology and control [7, 10, 14, 33, 35, 43, 44, 47, 56]. Paul and Bongard [42] found that even small changes to a robot's mass distribution have large effects on resultant gaits, leading to unique control-morphology pairs in evolved individuals. Later, Bongard [3] demonstrated the importance of morphology in the evolutionary process as a contributor to robust behavior in an individual. Further work by Cheney et al. [8] has demonstrated the emergence of coupling in soft robots, whose gaits are dependent upon body shape and size. Other researchers have explored the offloading of control to morphological elements [6, 21, 41, 44]. For example, Rieffel et al. [46] demonstrated distributed morphological control in a tensegrity system wherein the controller exploits the intrinsic dynamics of the body for effective locomotion. In this article, we investigate how the evolution of a high-level neurocontroller is affected by concurrently evolving joint-level controllers.

Hierarchical robot control distinguishes basic movement primitives from more complex behaviors [18]. The subsumption architecture, originally proposed by Brooks [4], ranks behaviors by order of precedence, and has proven effective in designing robotic controllers with multiple discrete behaviors [19, 29, 53, 54]. Subsumption-based control has been applied in many areas such as airplane control systems [26] and swarm robotics [39]. Recently, this technique has produced an evolved robot exhibiting locomotion, attacking, and fleeing behaviors in a single controller [30]. An alternative evolutionary method, proposed by Li and Miikkulainen [31], evolves subtask ANNs when fitness stagnates, producing a control system with small subunits whose execution is triggered by environmental conditions. The proposed ANN-DMM approach is also hierarchical, but low-level control is integrated with morphological aspects of joints (namely, the evolved configuration of muscle nodes), rather than part of the high-level controller.

Muscle-like actuators have previously been investigated in robotic systems using digital simulations. Lessin et al. [30] modeled muscles as spring systems with a variable spring constant. Changing the constant produces movements in the attached limbs. Simulated muscles also have the potential to create physically realistic gaits. Geijtenbeek et al. [22] demonstrated bipedal locomotion, driven by simulated muscles, on various terrains. Those muscles have defined attachment points, a contraction path, and activation dynamics defined through the optimization process. Wang et al. [57] found that muscle-driven bipedal gaits resulted in more upright postures and reduced energy expenditure. Unlike the models referred to above, which emulate muscles directly in simulation, the DMM is an abstract model characterized only by the positions of nodes around a joint and their responses to input signals. This approach enables the application of the DMM to motor-driven joints in robots while also contributing to the study of biological systems.

3 Methods

3.1 Digital Muscle Model

Biological muscles provide the power necessary for organisms to move and interact with their environment [17]. Working in antagonistic pairs, muscles allow for flexion and extension of individual joints, coordinated by neural systems [15, 16]. Although in some cases movement may appear to occur within a single degree of freedom (DOF) (for example, a knee extending), multiple muscles work together to both move and stabilize a joint. The DMM provides a means to manipulate joints by emulating the fundamental properties of biological muscles, while still being suitable for realization using conventional robotic actuators. Aspects of both control and morphology are integrated in the model, allowing for both to be evolved concurrently.

Figure 1 depicts a simple example of the DMM controlling one joint. Movement of the lower limb segment is controlled by four artificial muscles termed nodes. In this example, all nodes in a muscle group receive the same signal from a controller, with the activation function of each node determining its behavior. The location and activation function of each node are evolved. The combined responses of the muscle nodes, along with their positions, determine the behavior of the joint. In Figure 1 the muscle nodes are equally spaced around the joint, but in general this would not be the case.

Figure 1. 

A digital muscle group is composed of muscle nodes, radially distributed around a joint on a 2D plane. Conceptually, nodes exert a pulling force, drawing the limb segment towards the node's position on the plane. Antagonistic relationships (in terms of their relative positions and activation functions) can evolve between nodes, leading to coordinated movement. The outputs of a digital muscle group dictate the movement of a joint.

Figure 1. 

A digital muscle group is composed of muscle nodes, radially distributed around a joint on a 2D plane. Conceptually, nodes exert a pulling force, drawing the limb segment towards the node's position on the plane. Antagonistic relationships (in terms of their relative positions and activation functions) can evolve between nodes, leading to coordinated movement. The outputs of a digital muscle group dictate the movement of a joint.

Control The activation function of a digital muscle node governs when and how strongly it contracts, that is, pulls on a limb segment. This pulling force determines how far and fast a limb segment moves toward the node's location relative to the joint. Activation functions can be any function that maps an input signal value to an output value. For this study, activation functions are Gaussians with evolvable parameters: μ (center), σ2 (spread), and α (magnitude). Nodes are limited to positive exertion values, similarly to the function of natural muscles, which are only capable of active contraction. Consequently, at least two nodes, aligned as an antagonistic pair, are necessary to realize both flexion and extension in a joint. The activation functions of the nodes in a muscle group collectively define the response of the joint to an input signal. Activation functions for a sample group with four nodes can be seen in Figure 2. In the figure, an input signal value of −0.5 results in nodes 0, 1, and 2 exhibiting activation values of 0.77, 0.42, and 0.28, respectively. The activation value for node 3 is 0.

Figure 2. 

Activation functions for four nodes in a muscle group. An input signal determines the response of nodes according to the Gaussian activation function for each. The values of the activation functions for an input signal of −0.5 are highlighted.

Figure 2. 

Activation functions for four nodes in a muscle group. An input signal determines the response of nodes according to the Gaussian activation function for each. The values of the activation functions for an input signal of −0.5 are highlighted.

3.1.1 Morphology

Figure 3 shows a top-down view of the spatial component of muscle nodes, namely, where they are located with respect to their associated joint. Each node has a parameter defining its position on a unit circle around the joint. This position determines which direction the limb will be pulled in when a node contracts. Evolved node positions may produce a joint with a wide range of motion, as in a human shoulder, or one with limitations, as in a knee joint. If the nodes were aligned as shown in Figure 3, movement of the limb would be away from node 3, which has an activation value of 0 for inputs less than −0.1.

Figure 3. 

A top-down view of an individual muscle group consisting of four nodes placed radially around a joint. Each node has both an activation function and a spatial component. Together, these determine the strength and direction of pull placed on a joint by the individual node.

Figure 3. 

A top-down view of an individual muscle group consisting of four nodes placed radially around a joint. Each node has both an activation function and a spatial component. Together, these determine the strength and direction of pull placed on a joint by the individual node.

3.1.2 Joint Behavior

Joint behavior is calculated by combining the activation outputs, for a given input value, from all nodes in a group. The outputs for each node are projected into two values, one for each axis of the 2-DOF joint, according to the activation output and (x,y) coordinates of each node in a group. Figure 4 shows the results of aggregating the muscle node activations plotted in Figure 2 with the spatial positions of Figure 3; both the activation function of each node and its spatial location contribute to the response of the joint. That is to say, the response is an emergent property of the model, rather than directly dictated by the specific activation function or a single evolved parameter. Joint movement speeds are calculated for each joint as the difference between the current and desired angle divided by the simulation time step.

Figure 4. 

Activations for the 2-DOF joint controlled by a muscle group. The node activation functions depicted in Figure 2 map to the two desired joint positions seen in this figure. The humps in both curves near 0.4 are the result of node 3, which is a Gaussian function with μ = 0.4. Nodes for this muscle group are radially distributed at 45°, 135°, 225°, and 315° as in Figure 3.

Figure 4. 

Activations for the 2-DOF joint controlled by a muscle group. The node activation functions depicted in Figure 2 map to the two desired joint positions seen in this figure. The humps in both curves near 0.4 are the result of node 3, which is a Gaussian function with μ = 0.4. Nodes for this muscle group are radially distributed at 45°, 135°, 225°, and 315° as in Figure 3.

Figure 5 shows the mapping of a high-level control signal (in this case a simple sinusoidal wave) to the response of a hip joint in a quadruped. A muscle group composed of four nodes controls each joint in the animat. Each muscle group receives a single control signal, which is distributed to the four nodes. The high-level controller needs only to provide one signal per joint, rather than unique signals for each muscle node.

Figure 5. 

Example of an input signal being converted to joint commands in the rear hip of a quadrupedal animat. (a) A signal (in this case a simple sinusoid) from a high-level controller is distributed to each muscle group (b) and is then passed to all nodes in the group (c). Each node takes the input value and determines its output by finding the point on the Gaussian indicated by the input. This output is then combined with the spatial position of the node to determine the output for each DOF. The outputs of all nodes in the joint are aggregated (d) to derive the two motor movement commands for a joint in a robot (e). These commands are then sent to the motors (f) associated with the joint.

Figure 5. 

Example of an input signal being converted to joint commands in the rear hip of a quadrupedal animat. (a) A signal (in this case a simple sinusoid) from a high-level controller is distributed to each muscle group (b) and is then passed to all nodes in the group (c). Each node takes the input value and determines its output by finding the point on the Gaussian indicated by the input. This output is then combined with the spatial position of the node to determine the output for each DOF. The outputs of all nodes in the joint are aggregated (d) to derive the two motor movement commands for a joint in a robot (e). These commands are then sent to the motors (f) associated with the joint.

3.2 Simulation Environment

Simulations are conducted with the Open Dynamics Engine (ODE) [50], a 3D rigid-body physics simulation environment. ODE handles collisions between rigid bodies, friction, and gravity that act upon the components. For all experiments described in this article, simulations span 10 s of simulation time with a time step of 0.005 s. The environment is a flat, high-friction surface resulting in minimal slippage between the animat and substrate.

3.3 Quadrupedal Animat

Figure 6 shows the quadrupedal animat used in this study. The torso is 3 units long, 1 unit wide, and 0.5 units tall, comprising 45% of the total mass of the animat. Each leg is connected to the main body by a hip joint and divided into an upper and lower segment connected by a knee joint. Upper legs are 0.5 units long with a radius of 0.1 units and 9% of the total body mass per segment. The lower leg segments are 0.75 units long with a radius of 0.1 units and comprise 4.75% of the total body mass each. Joints are modeled as 2-DOF hinges, allowing for movement along and away from the torso with a ±120° range of motion on each axis.

Figure 6. 

The quadrupedal animat used in this study has eight 2-DOF joints. Each leg has a hip and a knee joint. Morphological dimensions and masses of the components are fixed.

Figure 6. 

The quadrupedal animat used in this study has eight 2-DOF joints. Each leg has a hip and a knee joint. Morphological dimensions and masses of the components are fixed.

3.4 Evolutionary Setup

As described in Sections 4, 5, and 6, we conducted evolutionary experiments with DMM-only, ANN-only, and ANN-DMM controllers. For all experiments, 20 replicate runs were executed, each initialized with a unique randomly generated seed with an evolutionary time of 2000 generations and population of 120 individuals. Fitness is the Euclidean distance from the starting location to the center of the torso after 10 s of simulation time. DMM-only or DMM components of a controller comprise eight muscle groups with four nodes per group. Muscle nodes are initially evenly distributed around the joint with randomly initialized parameters for the activation function. For purposes of crossover, recombination treats each muscle group as a complete unit, creating a child genome that is a composition of selected muscle groups from the two parents. Each muscle group maps to a joint in the quadrupedal animat. During crossover, a child individual is created from two parents, with muscle groups assigned to the corresponding joints. Genomes for DMM-only and DMM components consist of 128 parameters (8 muscle groups × 4 nodes × (3 Gaussian + 1 position parameter)), resulting in an average of 6 mutations per genome. Individual parameters are mutated with a normal distribution around the current value with an approximate range of ±10% of the parameter value.

In the initial DMM-only controller experiments, evolution was conducted using a genetic algorithm. The next generation is populated with two-way tournament selection. Elitism is not used. Crossover and mutation are applied at each generation with 10% and 5% probabilities, respectively.

For the subsequent experiments involving ANN-only and hybrid ANN-DMM controllers, we used the NEAT algorithm [51] to evolve the ANN component of the controller. NEAT produces recurrent artificial neural networks (ANNs) with a genetic algorithm. Starting with fully connected input-to-output ANNs, the algorithm complexifies networks by adding hidden nodes and connections over evolutionary time. Moreover, speciation provides principled crossover between related ANNs. NEAT parameters used in these experiments are listed in Table 1. These parameters were derived from preliminary experiments and found to evolve high-performing individuals in various locomotion tasks.

Table 1. 

NEAT parameters for gait evolution.

ParameterValue
Compatibility threshold 5.0 
Young age threshold 15 
Old age threshold 35 
Minimum species 
Maximum species 25 
Recurrent probability 0.25 
Crossover rate 0.75 
Mutation rate 0.33 
Mutate weights probability 0.90 
Weight mutation rate 0.75 
Max weight 20 
Add neuron probability 0.04 
Add link probability 0.10 
Remove link probability 0.10 
ParameterValue
Compatibility threshold 5.0 
Young age threshold 15 
Old age threshold 35 
Minimum species 
Maximum species 25 
Recurrent probability 0.25 
Crossover rate 0.75 
Mutation rate 0.33 
Mutate weights probability 0.90 
Weight mutation rate 0.75 
Max weight 20 
Add neuron probability 0.04 
Add link probability 0.10 
Remove link probability 0.10 

Twenty replicate runs per treatment are deployed onto a server with 64 cores running at 2.1 GHz and 64 GB of RAM. A replicate's simulations are parallelized per generation using the Python multiprocessing package. Evolutionary runs entail 2000 generations with a population of 120 individuals, resulting in 240,000 evaluations per replicate. Approximate running time for each replicate is 6 h.

4 Evolving Low-Level Control

For the purposes of comparison, we began by evolving locomotion in DMM-based animats where the high-level “control” is a simple oscillating signal,
formula
where t represents the current simulation time. These experiments provide insight into the capabilities of DMM-based control in the evolution of four-legged gaits. They are similar to the experiments described in [37], except that the simulation time has been increased from 5 s to 10 s and the evolutionary time reduced from 12,000 generations to 2,000 generations, aligning the parameters with those used in the ANN-only and ANN-DMM experiments presented later in this article.

We observed the evolution of several distinct locomotion strategies across the replicates, including bounding, shuffling, leaping, and running gaits, exemplified in Figure 7. Figure 7a shows a rear-legged bounding or galloping gait that travels 25.8 units during a simulation. The forelimbs balance the animat and generate forward propulsion, but the rear limbs provide the majority of propulsive force. A brisk walking gait is shown in Figure 7b, traveling 26.94 units. Figure 7c illustrates a sideways bounding gait, which travels 29.97 units over the evaluation period. The diversity of evolved behaviors highlights the expressive capability of the DMM. Videos of the evolved gaits are available at http://y2u.be/_zaS4_2r-eU.

Figure 7. 

Sample gaits for evolved digital-muscle-based animats. (a) Rear-legged bounding. (b) Walking. (c) Sideways bounding.

Figure 7. 

Sample gaits for evolved digital-muscle-based animats. (a) Rear-legged bounding. (b) Walking. (c) Sideways bounding.

As in our previous study [37], we observed the evolution of characteristics reminiscent of animal joints and their control, including symmetric rear hip movements for an individual with a rear bounding gait and kneelike functional specialization. For a discussion of those results, we refer the reader to [37].

In the remainder of this article, we focus on the performance of evolved solutions and the characteristics of the resultant ANNs. Figure 8 plots the performance across twenty replicate runs of evolved DMM-only controllers in the quadrupedal animat. The farthest-traveling individuals cover an average of 24.5 units over the simulation. The most effective individuals exceed 30 units, exhibiting three-legged bounding (34.55 units) or shuffling (30.90 units) gaits.

Figure 8. 

Average maximum fitness and average mean fitness across twenty replicate runs per generation for quadruped animats with DMM controllers. Shaded regions represent the 95% confidence intervals.

Figure 8. 

Average maximum fitness and average mean fitness across twenty replicate runs per generation for quadruped animats with DMM controllers. Shaded regions represent the 95% confidence intervals.

5 Combining High- and Low-Level Controllers

In this section, we investigate the integration of low-level DMM-based joints and high-level control via ANNs; both evolve concurrently. We are primarily interested in the following questions: Do ANN-DMM hybrid controllers outperform their ANN-only and DMM-only counterparts? What differences, if any, arise between various hybrid ANN-DMM configurations in overall performance and ANN complexity with regard to the numbers of connections and nodes in the networks?

5.1 Integrating ANN with DMM

Evolving a hybrid controller requires linking the outputs of the ANN to the inputs of the DMM. We investigate two strategies, illustrated in Figure 9, for connecting ANN to DMM. The top part of the figure shows the singly connected (SC) architecture, where the ANN has a single output for each joint. This configuration is similar to the DMM-only controllers described in Section 3.1, where all muscle nodes associated with a joint share the same ANN output. Movement is determined by combining the individual responses of each node to the signal. The bottom part shows the individually connected (IC) configuration, where a unique ANN output is connected to each node of a muscle group. The IC strategy potentially allows for more fine-grained control of the individual muscle nodes by the ANN, but at a cost of an increased number of ANN outputs. In the example shown in Figure 9, IC ANNs require four times as many outputs as the SC ANNs.

Figure 9. 

Examples illustrating the two connection strategies tested in this study: (top) singly connected (SC) and (bottom) individually connected (IC). Interaction between the ANN and DMM-based joint proceeds as follows: (a) ANN receives input from sensors and produces output(s): one for an SC joint and four for an IC joint. (b) For an SC joint, the same ANN output signal is distributed to each of four muscle nodes. For an IC joint, each muscle node receives its own signal directly from the ANN. (c) The position and activation function of a muscle node determines its response to the incoming signal. (d) Responses of the muscle nodes are combined and (e) passed to the platform.

Figure 9. 

Examples illustrating the two connection strategies tested in this study: (top) singly connected (SC) and (bottom) individually connected (IC). Interaction between the ANN and DMM-based joint proceeds as follows: (a) ANN receives input from sensors and produces output(s): one for an SC joint and four for an IC joint. (b) For an SC joint, the same ANN output signal is distributed to each of four muscle nodes. For an IC joint, each muscle node receives its own signal directly from the ANN. (c) The position and activation function of a muscle node determines its response to the incoming signal. (d) Responses of the muscle nodes are combined and (e) passed to the platform.

5.2 NEAT Modifications for ANN-DMM Coupling

Since NEAT handles ANN evolution only, tracking is required to apply crossover to the corresponding DMM component. We extended the NEAT algorithm with a genome identifier, used to match the two components of the genome (ANN, DMM) in the evolutionary process. During crossover, the ANNs are recombined within the NEAT algorithm. For the DMM components, the two parent genomes are identified by the genome IDs, and crossover is then applied to the DMM components, using the method of crossover described in Section 3.4, to form the full ANN-DMM child genome. Mutation of the DMM is applied per generation to each individual at a rate of 5% per parameter; each joint comprises four muscle nodes with four parameters each.

5.3 Evolutionary Progression

We conducted three treatments for this study: ANN-only, singly connected ANN-DMM (denoted SC), and individually connected ANN-DMM (denoted IC). Figure 10 plots the evolutionary progression of the maximum fitness (distance traveled) across the 20 replicates for the three treatments. As shown, the hybrid ANN-DMM controllers outperform ANN-only control across replicate runs. IC controllers perform slightly better than SC controllers, with the confidence intervals often overlapping.

Figure 10. 

Average maximum fitness across 20 replicate runs per treatment in the quadruped platform. Shaded areas represent the 95% confidence intervals.

Figure 10. 

Average maximum fitness across 20 replicate runs per treatment in the quadruped platform. Shaded areas represent the 95% confidence intervals.

Figure 11 presents boxplots of the distributions of fitness for the farthest-traveling individual from each replicate run. The two hybrid controllers travel farther than their ANN-only counterparts. Indeed, a Wilcoxon rank-sum test, used here and throughout the rest of the article, indicates that the values for the hybrid controllers are both significantly higher than the ANN-only (p < 0.0001), while there is no significant difference in performance between the two hybrid controllers (p = 0.2828).

Figure 11. 

Distribution of fitnesses for the best individual per replicate across the three controllers. Results are significantly different between the hybrid and ANN-only controllers. There is no significant difference between the SC and IC controllers.

Figure 11. 

Distribution of fitnesses for the best individual per replicate across the three controllers. Results are significantly different between the hybrid and ANN-only controllers. There is no significant difference between the SC and IC controllers.

5.4 ANN Analysis

In addition to performance, we are also interested in characteristics of the evolved ANNs, specifically the numbers of internal connections and hidden nodes. Differences in the structure of the ANNs could indicate that the ANN is offloading control functionality to the low-level DMM controller. Figure 12 plots the number of connections in the evolved networks. From Figures 11 and 12, we observe that the SC controllers consistently have fewer connections than IC controllers while exhibiting similar performance; indeed, IC controllers have more than twice as many connections as SC.

Figure 12. 

Distribution of the number of connections in the best individual per replicate across the three treatments. The three treatments are all significantly different from each other (p < 0.0001).

Figure 12. 

Distribution of the number of connections in the best individual per replicate across the three treatments. The three treatments are all significantly different from each other (p < 0.0001).

While the number of network connections across the three treatments is significantly different, the number of network nodes (neurons) is similar. Figure 13 shows the distribution of neurons across the best individuals. Unlike the number of connections, there is not a clear separation between the different treatments. We speculate that this outcome could be due to the different roles between connections (information transfer) and hidden nodes (units of computation) in ANNs [59]. Although high-performing, IC controllers may require greater information flow in the evolved networks to compensate for the increased connectivity between ANN and DMM. The number of connections is also influenced by the number of ANN inputs and outputs in each treatment. For the evolved controllers, the SC treatment (22 inputs, 8 outputs) has the fewest inputs, outputs, and initial connections, while the ANN-only (22 inputs, 16 outputs) is second, and the IC (22 inputs, 32 outputs) has the most.

Figure 13. 

Distribution of the number of neurons in the best individual per replicate across the three treatments. There is no significant difference among the three (p = 0.1589 for ANN/SC, p = 0.1437 for ANN/IC, and p = 0.8923 for SC/IC).

Figure 13. 

Distribution of the number of neurons in the best individual per replicate across the three treatments. There is no significant difference among the three (p = 0.1589 for ANN/SC, p = 0.1437 for ANN/IC, and p = 0.8923 for SC/IC).

5.5 Summary

Hybrid controllers exhibit higher fitness than ANN-only controllers in locomotion tasks. Additionally, SC controllers have fewer connections than ANN-only controllers in the evolved networks. This result appears to coincide with theories of control in biological organisms, where movement “primitives” in the spinal cord are thought to govern the coordination of multiple muscles, simplifying the high-level commands dictating locomotion [24]. With regard to ANN complexity, SC controllers have the fewest connections, while there is no significant difference in the number of neurons among the three treatments.

Next, we explore further the characteristics of the evolved ANNs, specifically, investigating to what extent they can be simplified without sacrificing fitness.

6 Phased Searching

We observe that the ANNs evolved in the previous section are rather large, comprising hundreds of connections and tens of hidden nodes. Although complexification is an important feature of the NEAT algorithm, unchecked growth could produce bloated networks. To combat this problem in ANNs, pruning [48] and phased searching [25, 52] apply simplification operations during the evolutionary process. Results have been promising, showing that not only are the resultant networks less complex, but they also sometimes achieve higher fitnesses [48]. In this study, the addition of the DMM component potentially offloads some control functionality from the ANN, as we observed in the number of connections in the previous section. Therefore, we employ the phased search strategy introduced by Green [25] to explore whether and how ANN characteristics change if this simplification strategy is applied in the evolution of hybrid ANN-DMM controllers. We examine the structure of the evolved networks with respect to the number of connections and neurons in the evolved networks.

6.1 Phased Searching in NEAT

Phased searching [25] modifies the NEAT algorithm by switching between complexification and simplification phases. The process begins by specifying a mean population complexity (MPC), defined as the number of unique neuron and connection genes present in the population divided by the population size. The algorithm then proceeds in a complexifying phase, as in traditional NEAT, but switches to the simplification phase once the MPC value is reached. In the simplification phase, adding structure to the ANNs is prohibited in favor of removing redundant structure. Crossover is also disabled, as it can maintain structure in the population. Simplification continues until a low threshold value of MPC is reached, upon which complexification resumes.

6.2 Experimental Setup

Table 2 shows the three parameter configurations we explore for phased searching (Low, Mid, High) and continue with the three controller setups (ANN-only, SC, IC) from the previous section. The terms Low, Mid, and High describe the relative frequency of adding/removing neurons or links. Hence, nine new treatments are conducted. The NEAT parameters are identical to those of the previous experiment (Table 1) except for modifications to the node and connection mutation rates listed in Table 2.

Table 2. 

NEAT mutation parameters for phased searching.

ParameterBaseLowMidHigh
Add neuron probability 0.04 0.02 0.04 0.4 
Remove neuron probability 0.04 0.02 0.04 0.4 
Add link probability 0.1 0.05 0.1 0.4 
Remove link probability 0.1 0.05 0.1 0.4 
ParameterBaseLowMidHigh
Add neuron probability 0.04 0.02 0.04 0.4 
Remove neuron probability 0.04 0.02 0.04 0.4 
Add link probability 0.1 0.05 0.1 0.4 
Remove link probability 0.1 0.05 0.1 0.4 

6.3 Results

The evolutionary trajectories of the maximum fitness for the nine new treatments and the base treatment are shown in Figure 14. The ANN-DMM controllers again exhibit higher maximum fitnesses than ANN-only controllers across the three configurations. The difference is more noticeable for the High treatments. We speculate that the DMM could potentially be providing stability in joint control, compensating for the increased ANN mutation rates. In addition, we observe that the maximum fitnesses are slightly lower across all treatments than those observed in the base experiments; we shall discuss this issue later.

Figure 14. 

Evolutionary trajectory of the maximum fitness per controller configuration across replicates for the three parameter configurations and the base experiments described previously in Section 5. Shaded areas represent the 95% confidence intervals.

Figure 14. 

Evolutionary trajectory of the maximum fitness per controller configuration across replicates for the three parameter configurations and the base experiments described previously in Section 5. Shaded areas represent the 95% confidence intervals.

Figure 15 plots the average number of connections in the ANNs, over evolutionary time, for the three phased-search treatments. The hybrid ANN-DMM controllers decrease in size over evolutionary time; the rate of decline is most pronounced in the Mid and High parameter configurations. However, ANN-only networks continue to grow across all three parameter configurations. This result suggests that the DMM component evolves functionality complementary to that of the high-level ANN.

Figure 15. 

Evolutionary trajectory of the average number of connections in a population per controller configuration across replicates for the three parameter configurations. Shaded areas represent the 95% confidence intervals.

Figure 15. 

Evolutionary trajectory of the average number of connections in a population per controller configuration across replicates for the three parameter configurations. Shaded areas represent the 95% confidence intervals.

Figure 16 plots the average number of neurons in the ANNs for the three controller configurations. In all nine treatments, the average controller experiences an initial growth in the number of neurons. This is to be expected, as NEAT begins with fully connected input-to-output networks with no hidden nodes. Over time, the hybrid controllers exhibit decreases in the number of neurons for the Mid and High parameter configurations, while the ANN-only controllers continue to add structure.

Figure 16. 

Evolutionary trajectory of the average number of neurons in a population per controller configuration across replicates for the three parameter configurations. Shaded areas represent the 95% confidence intervals.

Figure 16. 

Evolutionary trajectory of the average number of neurons in a population per controller configuration across replicates for the three parameter configurations. Shaded areas represent the 95% confidence intervals.

As noted above, the phased searching strategy alternates between adding structure and refining the networks. Figure 17 shows the average time in each phase for the different evolutionary runs. The ANN-only treatments remain in the complexification phase for the majority of evolution, whereas the hybrid treatments spend more time in the simplification phase. We speculate that offloading to the DMM of functionality to govern basic locomotion may facilitate simplification in the ANN.

Figure 17. 

Proportion of time in simplification versus complexification phases per each treatment.

Figure 17. 

Proportion of time in simplification versus complexification phases per each treatment.

Whereas Figures 14, 15, and 16 show general trends in connections and neurons throughout the population, Figure 18 plots the the fitness distributions of the farthest-traveling individual per replicate. As with the results from Section 5, the hybrid ANN-DMM controllers exhibit significantly higher fitnesses than their ANN-only counterparts (p < 0.0001, Wilcoxon rank-sum test).

Figure 18. 

Fitness distribution of the farthest-traveling individual in each replicate across the three parameter configurations for phased searching.

Figure 18. 

Fitness distribution of the farthest-traveling individual in each replicate across the three parameter configurations for phased searching.

Figure 19 plots the number of connections in the best individual per treatment. While the ANN-only controllers have similar numbers of connections for each of the three parameter settings, both hybrid controllers exhibit decreasing network complexity across the three parameter configurations, especially in the High configuration. The SC controllers have fewer than 100 connections in the High parameter configuration. In short, the phased-search strategy produces ANNs with significantly lower complexity than unmodified NEAT (Figure 19 versus Figure 12), while maintaining high fitness (Figure 18 versus Figure 11) under the Mid and High parameter configurations.

Figure 19. 

Distribution of the number of connections in the ANNs for the farthest-traveling individual per replicate per each controller across the three parameter configurations.

Figure 19. 

Distribution of the number of connections in the ANNs for the farthest-traveling individual per replicate per each controller across the three parameter configurations.

Figure 20 plots the number of neurons in each network for the highest-fitness individuals per treatment. The SC controllers again contain the fewest neurons, while the ANN-only controllers grow as the mutation rates increase. IC controllers do not undergo the same reduction in neurons as they do for connections, perhaps due to the nature of the neurons (calculation) and connections (information transfer), as well as the high number of output neurons required to link ANN to DMM, as discussed previously.

Figure 20. 

Distribution of the number of neurons in the ANNs for the farthest-traveling individual per replicate per each controller across the three parameter configurations.

Figure 20. 

Distribution of the number of neurons in the ANNs for the farthest-traveling individual per replicate per each controller across the three parameter configurations.

7 Cross-Treatment Comparisons

Figure 21 shows the distributions of performance of the highest-fitness individual per replicate across all treatments conducted in this study. Notably, the highest-fitness individual, across all runs, is a DMM-only controller with a fixed periodic oscillating signal traveling 34.55 units. The highest-performing individuals with hybrid controllers are close behind, having fitnesses of 33.07 units for the High-SC strategy and 32.32 units for the High-IC strategy. As shown in the plot, DMM-only controllers also exhibit high variability in performance. SC and IC controllers exhibit less variation across the replicates, suggesting that perhaps the hybrid controllers are less prone to getting trapped by local optima during evolution. Of course, the ANN component enables a system to respond to sensed information (in our case, touch and joint angle sensors), unlike the DMM-only configuration. Yet this result indicates that the control needed for basic gaits can be relegated to the DMM. What is perhaps most surprising is the low performance of ANN-only controllers throughout the experiments, relative to their DMM-based counterparts.

Figure 21. 

Fitness distribution of the best individuals per treatment across the three experiments conducted in this study. Base refers to the DMM-only results from Section 4 and the ANN-DMM ones from Section 5. Low, Mid, and High refer to the phased searching results from Section 6.

Figure 21. 

Fitness distribution of the best individuals per treatment across the three experiments conducted in this study. Base refers to the DMM-only results from Section 4 and the ANN-DMM ones from Section 5. Low, Mid, and High refer to the phased searching results from Section 6.

8 Open Questions

In this study, hybrid ANN-DMM controllers outperform ANN-only controllers while exhibiting less complex high-level ANNs. These results are from a quadruped animat with eight joints. A remaining question is how the number of ANN inputs/outputs relates to ANN complexity. In other work [36], we have examined how the number of joints, and consequently the number of ANN outputs, affects ANN complexity in a robotic platform with 1 to 12 joints. There, we found that hybrid ANN-DMM controllers evolve less complex networks, in terms of the number of connections, as the number of joints in the robot increases, while maintaining similar performance. It remains an open question how the number of inputs affects ANN complexity.

The phased searching strategy often evolves ANN-DMM controllers with very sparse connectivity in the ANN, especially in the High configuration. In some cases, the number of connections results in ANNs that do not have any connectivity between certain inputs and outputs. This result suggests that some of the sensory information is not needed to drive locomotion. The relative simplicity of the environment is likely also a contributing factor in this article. Controllers are evolved on a flat, high-friction surface with no obstacles. Furthermore, we have also shown that DMM is capable of evolving effective quadrupedal locomotion when driven by a simple oscillating signal. These two factors, flat terrain and low-level DMM controller, might explain why the high-level controller evolves toward a relatively simple configuration. Adding terrain variation would require sensory processing/response not currently needed, likely resulting in more complex ANNs.

9 Conclusions

We have investigated a model of joint-level control inspired by that of biological organisms yet applicable to the control of robotic systems. Prior experiments showed that evolved DMM-based systems exhibit effective gaits in a quadrupedal animat, even when driven by a simple periodic oscillating signal acting as a high-level controller. However, in such a system the movements are essentially hardwired. Our primary focus in this article is evolving DMM-based joints concurrently with a high-level ANN controller, enabling the system to respond dynamically to sensed information.

The resulting hybrid ANN-DMM controllers (and even the DMM-only controllers) consistently outperform their ANN-only counterparts in terms of distance traveled. The two connection strategies evaluated, singly connected and individually connected, exhibit similar fitnesses, but the former produces less complex ANNs in both number of neurons and number of connections. In all cases, however, the evolved networks are quite large, with tens of neurons and hundreds of connections. Phased searching drastically reduces network complexity with only a small reduction in performance. Given these results, it appears the DMM compensates for a less complex high-level controller by governing basic movements, at least on flat terrain. A possible implication is that a high-level controller is free to focus on other matters, including environmental dynamics that affect low-level control, as well as more implementation of complex behaviors. In future work, we plan to investigate the evolution of hybrid controllers in varied terrain and dynamic environments, which might engage the sensory capabilities of an ANN more than simple locomotion tasks.

Appendix: Additional Materials

Source code for reproducing the experiments in this study can be found at https://github.com/jaredmoore/ANN_DMM_Quadruped. The repository includes the DMM controller code, a modified version of the MultiNEAT package [9], code for each of the treatments, and Bash scripts for launching replicate runs. Instructions for configuring the ODE physics engine are also provided.

Acknowledgments

This research was conducted while J.M.M. was a postdoctoral researcher at the NSF BEACON Center at Michigan State University. The authors gratefully acknowledge the contributions and feedback provided by Anthony Clark, Xiaobo Tan, Craig McGowan, and members of the BEACON Center at Michigan State University. This work was supported in part by National Science Foundation grants CNS-1059373, CNS-0915855, and DBI-0939454, and by a grant from Michigan State University.

References

1
Alexander
,
R.
, &
Vernon
,
A.
(
1975
).
The mechanics of hopping by kangaroos (Macropodidae)
.
Journal of Zoology
,
177
(
2
),
265
303
.
2
Autumn
,
K.
,
Sitti
,
M.
,
Liang
,
Y. A.
,
Peattie
,
A. M.
,
Hansen
,
W. R.
,
Sponberg
,
S.
,
Kenny
,
T. W.
,
Fearing
,
R.
,
Israelachvili
,
J. N.
, &
Full
,
R. J.
(
2002
).
Evidence for van der Waals adhesion in gecko setae
.
Proceedings of the National Academy of Sciences of the U.S.A.
,
99
(
19
),
12252
12256
.
3
Bongard
,
J.
(
2011
).
Morphological change in machines accelerates the evolution of robust behavior
.
Proceedings of the National Academy of Sciences of the U.S.A.
,
108
(
4
),
1234
1239
.
4
Brooks
,
R. A.
(
1986
).
A robust layered control system for a mobile robot
.
IEEE Journal of Robotics and Automation
,
2
(
1
),
14
23
.
5
Brooks
,
R. A.
(
1989
).
A robot that walks; emergent behaviors from a carefully evolved network
.
Neural Computation
,
1
(
2
),
253
262
.
6
Bruce
,
J.
,
Caluwaerts
,
K.
,
Iscen
,
A.
,
Sabelhaus
,
A.
, &
SunSpiral
,
V.
(
2014
).
Design and evolution of a modular tensegrity robot platform
. In
Proceedings of the 2014 IEEE International Conference on Robotics and Automation
(pp.
3483
3489
).
New York
:
IEEE Press
.
7
Chaumont
,
N.
,
Egli
,
R.
, &
Adami
,
C.
(
2007
).
Evolving virtual creatures and catapults
.
Artificial Life
,
13
(
2
),
139
157
.
8
Cheney
,
N.
,
MacCurdy
,
R.
,
Clune
,
J.
, &
Lipson
,
H.
(
2013
).
Unshackling evolution: Evolving soft robots with multiple materials and a powerful generative encoding
. In
Proceedings of the 15th Annual Conference on Genetic and Evolutionary Computation
(pp.
167
174
).
New York
:
ACM
.
9
Chervenski
,
P.
(
2016
).
MultiNEAT: Portable neuroevolution library
. https://github.com/peter-ch/MultiNEAT.
10
Chiel
,
H. J.
, &
Beer
,
R. D.
(
1997
).
The brain has a body: Adaptive behavior emerges from interactions of nervous system, body and environment
.
Trends in Neurosciences
,
20
(
12
),
553
557
.
11
Cliff
,
D.
,
Husbands
,
P.
, &
Harvey
,
I.
(
1993
).
Explorations in evolutionary robotics
.
Adaptive Behavior
,
2
(
1
),
73
110
.
12
Clune
,
J.
,
Beckmann
,
B. E.
,
Ofria
,
C.
, &
Pennock
,
R. T.
(
2009
).
Evolving coordinated quadruped gaits with the HyperNEAT generative encoding
. In
Proceedings of the IEEE Congress on Evolutionary Computation
(pp.
2764
2771
).
New York
:
IEEE Press
.
13
Cully
,
A.
, &
Mouret
,
J.-B.
(
2013
).
Behavioral repertoire learning in robotics
. In
Proceedings of the 2013 ACM Genetic and Evolutionary Computing Conference
(pp.
175
182
).
New York
:
ACM
.
14
Cussat-Blanc
,
S.
, &
Pollack
,
J.
(
2014
).
Cracking the egg: Virtual embryogenesis of real robots
.
Artificial Life
,
20
(
3
),
361
383
.
15
Daley
,
M. A.
,
Felix
,
G. A.
, &
Biewener
,
A. A.
(
2007
).
Running stability is enhanced by a proximo-distal gradient in joint neuromechanical control
.
The Journal of Experimental Biology
,
210
(
3
),
383
394
.
16
Daley
,
M. A.
,
Voloshina
,
A.
, &
Biewener
,
A. A.
(
2009
).
The role of intrinsic muscle mechanics in the neuromuscular control of stable running in the guinea fowl
.
Journal of Physiology
,
587
(
11
),
2693
2707
.
17
Dickinson
,
M. H.
,
Farley
,
C. T.
,
Full
,
R. J.
,
Koehl
,
M. A. R.
,
Kram
,
R.
, &
Lehman
,
S.
(
2000
).
How animals move: An integrative view
.
Science
,
288
(
5463
),
100
106
.
18
Digney
,
B. L.
(
1996
).
Emergent hierarchical control structures: Learning reactive/hierarchical relationships in reinforcement environments
. In
Proceedings of the Fourth International Conference on Simulation of Adaptive Behavior
(pp.
363
372
).
Cambridge, MA
:
MIT Press
.
19
Fischer
,
V.
, &
Hickinbotham
,
S.
(
2011
).
Evolving a metabolic subsumption architecture for cooperative control of the e-puck
.
Memetic Computing
,
3
(
4
),
231
244
.
20
Floreano
,
D.
,
Husbands
,
P.
, &
Nolfi
,
S.
(
2008
).
Evolutionary robotics
. In
Handbook of Robotics
.
Berlin
:
Springer
.
21
Füchslin
,
R. M.
,
Dzyakanchuk
,
A.
,
Flumini
,
D.
,
Hauser
,
H.
,
Hunt
,
K. J.
,
Luchsinger
,
R. H.
,
Reller
,
B.
,
Scheidegger
,
S.
, &
Walker
,
R.
(
2013
).
Morphological computation and morphological control: Steps toward a formal theory and applications
.
Artificial Life
,
19
(
1
),
9
34
.
22
Geijtenbeek
,
T.
,
van de Panne
,
M.
, &
van der Stappen
,
A. F.
(
2013
).
Flexible muscle-based locomotion for bipedal creatures
.
ACM Transactions on Graphics
,
32
(
6
),
1
11
.
23
Gerritsen
,
K. G.
,
van den Bogert
,
A.
,
Hulliger
,
M.
, &
Zernicke
,
R. F.
(
1998
).
Intrinsic muscle properties facilitate locomotor control—a computer simulation study
.
Motor Control
,
2
(
3
),
206
220
.
24
Giszter
,
S. F.
,
Mussa-lvaldi
,
A.
, &
Bizzi
,
E.
(
1993
).
Convergent force fields organized in the frog's spinal cord
.
Journal of Neuroscience
,
13
(
2
),
467
491
.
25
Green
,
C.
(
2004
).
Phased searching with NEAT: Alternating between complexification and simplification
(
Technical report
).
26
Hartley
,
R.
, &
Pipitone
,
F.
(
1991
).
Experiments with the subsumption architecture
. In
Proceedings of the 1991 IEEE International Conference on Robotics and Automation
,
vol. 2
(pp.
1652
1658
).
New York
:
IEEE Press
.
27
Ijspeert
,
A. J.
,
Crespi
,
A.
, &
Cabelguen
,
J.-M.
(
2005
).
Simulation and robotics studies of salamander locomotion
.
Neuroinformatics
,
3
(
3
),
171
195
.
28
Inoue
,
K.
,
Sumi
,
T.
, &
Ma
,
S.
(
2007
).
CPG-based control of a simulated snake-like robot adaptable to changing ground friction
. In
Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems
(pp.
1957
1962
).
New York
:
IEEE Press
.
29
Koza
,
J. R.
(
1994
).
Evolution of a subsumption architecture that performs a wall following task for an autonomous mobile robot
. In
S. J.
Hanson
,
T.
Petsche
,
M.
Kearns
, &
R. L.
Rivest
(Eds.),
Proceedings of the Workshop on Computational Learning Theory and Natural Learning Systems: Intersections between theory and experiment
,
vol. 2
(pp.
321
346
).
Cambridge, MA
:
MIT Press
.
30
Lessin
,
D.
,
Fussell
,
D.
, &
Miikkulainen
,
R.
(
2013
).
Open-ended behavioral complexity for evolved virtual creatures
. In
Proceedings of the 2013 ACM Genetic and Evolutionary Computing Conference
(pp.
335
342
).
New York
:
ACM
.
31
Li
,
X.
, &
Miikkulainen
,
R.
(
2014
).
Evolving multimodal behavior through subtask and switch neural networks
. In
H.
Sayama
,
J.
Rieffel
,
S.
Risi
,
R.
Doursat
, &
H.
Lipson
(Eds.),
Artificial Life 14, Proceedings of the Fourteenth International Conference on the Synthesis and Simulation of Living Systems
.
Cambridge, MA
:
MIT Press
.
32
Lipson
,
H.
(
2005
).
Evolutionary robotics and open-ended design automation
. In
B.
Cohen
(Ed.),
Biomimetics
(pp.
129
155
).
Boca Raton, FL
:
CRC Press
.
33
Lipson
,
H.
, &
Pollack
,
J. B.
(
2000
).
Automatic design and manufacture of robotic lifeforms
.
Nature
,
406
(
6799
),
974
978
.
34
Matsuoka
,
K.
(
1987
).
Mechanisms of frequency and pattern control in the neural rhythm generators
.
Biological Cybernetics
,
56
,
345
353
.
35
Mautner
,
C.
, &
Belew
,
R.
(
2000
).
Evolving robot morphology and control
.
Artificial Life and Robotics
,
4
,
130
136
.
36
Moore
,
J. M.
(
2015
).
Exploring joint-level control in evolutionary robotics
.
PhD thesis, Michigan State University
.
37
Moore
,
J. M.
, &
McKinley
,
P. K.
(
2014
).
Evolving joint-level control with digital muscles
. In
Proceedings of the 2014 ACM Genetic and Evolutionary Computing Conference
(pp.
209
216
).
New York
:
ACM
.
38
Moore
,
J. M.
, &
McKinley
,
P. K.
(
2014
).
Investigating modular coupling of morphology and control with digital muscles
. In
Proceedings of the 14th International Conference on the Simulation and Synthesis of Living Systems
(pp.
148
155
).
New York
:
ACM
.
39
Nagata
,
F.
,
Otsuka
,
A.
,
Watanabe
,
K.
, &
Habib
,
M.
(
2014
).
Network-based subsumption architecture for broadcast control of multiple mobile robots based on a poor hardware/software platform
. In
Y.-T.
Kim
,
I.
Kobayashi
, &
E.
Kim
(Eds.),
Soft computing in advanced robotics
,
vol. 269
(pp.
1
17
).
Cham, Switzerland
:
Springer International
.
40
Nolfi
,
S.
, &
Floreano
,
D.
(
2000
).
Evolutionary robotics: The biology, intelligence and technology of self-organizing machines
.
Cambridge, MA
:
MIT Press
.
41
Paul
,
C.
(
2006
).
Morphological computation: A basis for the analysis of morphology and control requirements
.
Robotics and Autonomous Systems
,
54
(
8
),
619
630
.
42
Paul
,
C.
, &
Bongard
,
J. C.
(
2001
).
The road less travelled: Morphology in the optimization of biped robot locomotion
. In
Proceedings of the 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems
(pp.
226
232
).
New York
:
IEEE Press
.
43
Pen
,
J.
,
Caarls
,
W.
,
Wisse
,
M.
, &
Babuska
,
R.
(
2013
).
Evolutionary co-optimization of control and system parameters for a resonating robot arm
. In
Proceedings of the 2013 IEEE International Conference on Robotics and Automation
(pp.
4195
4202
).
New York
:
IEEE Press
.
44
Pfeifer
,
R.
,
Lungarella
,
M.
, &
Iida
,
F.
(
2007
).
Self-organization, embodiment, and biologically inspired robotics
.
Science
,
318
(
5853
),
1088
1093
.
45
Reil
,
T.
, &
Husbands
,
P.
(
2002
).
Evolution of central pattern generators for bipedal walking in a real-time physics environment
.
IEEE Transactions on Evolutionary Computation
,
6
(
2
),
159
168
.
46
Rieffel
,
J. A.
,
Valero-Cuevas
,
F. J.
, &
Lipson
,
H.
(
2010
).
Morphological communication: Exploiting coupled dynamics in a complex mechanical structure to achieve locomotion
.
Journal of the Royal Society Interface
,
7
(
45
),
613
621
.
47
Schramm
,
L.
,
Jin
,
Y.
, &
Sendhoff
,
B.
(
2011
).
Emerged coupling of motor control and morphological development in evolution of multi-cellular animats
. In
G.
Kampis
,
I.
Karsai
, &
E.
Szathmáry
(Eds.),
Proceedings of the 10th European Conference on Advances in Artificial Life: Darwin meets von Neumann
(pp.
27
34
).
Berlin, Heidelberg
:
Springer
.
48
Siebel
,
N. T.
,
Botel
,
J.
, &
Sommer
,
G.
(
2009
).
Efficient neural network pruning during neuro-evolution
. In
Proceedings of the 2009 International Joint Conference on Neural Networks
(pp.
2920
2927
).
New York
:
IEEE Press
.
49
Sims
,
K.
(
1994
).
Evolving virtual creatures
. In
Proceedings of the 21st Annual Conference on Computer Graphics and Interactive Techniques
(pp.
15
22
).
New York
:
ACM
.
50
Smith
,
R.
(
2013
).
Open Dynamics Engine
. .
51
Stanley
,
K. O.
, &
Miikkulainen
,
R.
(
2002
).
Evolving neural networks through augmenting topologies
.
Evolutionary Computation
,
10
(
2
),
99
127
.
52
Tan
,
M.
,
Deklerck
,
R.
,
Cornelis
,
J.
, &
Jansen
,
B.
(
2013
).
Phased searching with NEAT in a time-scaled framework: Experiments on a computer-aided detection system for lung nodules
.
Artificial Intelligence Medicine
,
59
(
3
),
157
167
.
53
Togelius
,
J.
(
2004
).
Evolution of a subsumption architecture neurocontroller
.
Journal of Intelligent and Fuzzy Systems
,
15
(
1
),
15
20
.
54
Turner
,
J.
,
Givigi
,
S.
, &
Beaulieu
,
A.
(
2013
).
Implementation of a subsumption based architecture using model-driven development
. In
Proceedings of the 2013 IEEE International Systems Conference
(pp.
331
338
).
New York
:
IEEE Press
.
55
Valero-Cuevas
,
F.
,
Yi
,
J.-W.
,
Brown
,
D.
,
McNamara
,
R.
,
Paul
,
C.
, &
Lipson
,
H.
(
2007
).
The tendon network of the fingers performs anatomical computation at a macroscopic scale
.
IEEE Transactions on Biomedical Engineering
,
54
(
6
),
1161
1166
.
56
von Haller
,
B.
,
Ijspeert
,
A.
, &
Floreano
,
D.
(
2005
).
Co-evolution of structures and controllers for Neubot underwater modular robots
. In
M. S.
Capcarrere
,
A. A.
Freitas
,
P. J.
Bentley
,
C. G.
Johnson
, &
J.
Timmis
(Eds.),
Advances in Artificial Life
(pp.
189
199
).
Berlin, Heidelberg
:
Springer
.
57
Wang
,
J. M.
,
Hamner
,
S. R.
,
Delp
,
S. L.
, &
Koltun
,
V.
(
2012
).
Optimizing locomotion controllers using biologically-based actuators and objectives
.
ACM Transactions on Graphics
,
31
(
4
),
25
.
58
Wolpert
,
D. M.
, &
Ghahramani
,
Z.
(
2000
).
Computational principles of movement neuroscience
.
Nature Neuroscience
,
3
,
1212
1217
.
59
Yao
,
X.
(
1999
).
Evolving artificial neural networks
.
Proceedings of the IEEE
,
87
(
9
),
1423
1447
.

Author notes

Contact author.

∗∗

School of Computing and Information Systems, Grand Valley State University, Allendale, Michigan. E-mail: moore-jar@gvsu.edu.

Department of Computer Science and Engineering, Michigan State University, 428 South Shaw Lane, Room 3115, East Lansing, MI 48824. E-mail: mckinley@cse.msu.edu