Abstract

Completely soft and flexible robots offer to revolutionize fields ranging from search and rescue to endoscopic surgery. One of the outstanding challenges in this burgeoning field is the chicken-and-egg problem of body-brain design: Development of locomotion requires the preexistence of a locomotion-capable body, and development of a location-capable body requires the preexistence of a locomotive gait. This problem is compounded by the high degree of coupling between the material properties of a soft body (such as stiffness or damping coefficients) and the effectiveness of a gait. This article synthesizes four years of research into soft robotics, in particular describing three approaches to the co-discovery of soft robot morphology and control. In the first, muscle placement and firing patterns are coevolved for a fixed body shape with fixed material properties. In the second, the material properties of a simulated soft body coevolve alongside locomotive gaits, with body shape and muscle placement fixed. In the third, a developmental encoding is used to scalably grow elaborate soft body shapes from a small seed structure. Considerations of the simulation time and the challenges of physically implementing soft robots in the real world are discussed.

1 Introduction

Imagine a soft, resilient, and deformable robot able to change shape and squeeze through small apertures. The idea of using such a robot for urban search and rescue holds great appeal, particularly in light of recent earthquakes in China, New Zealand, and Japan. Once the domain of science fiction, soft robots are approaching reality—thanks to recent advances in engineering and materials science. Unfortunately, the very properties that make soft robots so appealing also introduce significant obstacles, especially in the domains of design and control. Elasticity and deformability come at the cost of resonances and tight dynamic coupling between components [24]—properties that are often assiduously avoided in conventional engineering approaches to robotic design. Small changes to the elasticity of a soft robot can cause unexpectedly large changes in performance.

The problems of soft robot design can be summarized with three questions:

  • What should a soft robot look like? (Morphology)

  • What should a soft robot be made out of? (Material)

  • How should a robot move? (Locomotion)

Of course, these are not independent variables: Solving each problem is predicated upon, and sensitive to, the preexistence of solutions to the other problems. The design of a soft robot's locomotive gait, for instance, depends upon both its morphology and properties such as elasticity and friction. This is in a sense an elaboration on the chicken-and-egg problem posed by body-brain design in more conventional robots [14, 15], with the added complexities that come from the effects of material properties upon a soft body's dynamics. In light of that, our approach to solving the problem will be similar: coevolution.

The choice of a stochastic search technique such as coevolution is justified in this case both by the enormous scope of the problem and by the absence of the purely analytical design methodologies available in conventional robotics. The field of evolutionary design has had considerable success in other complex design domains ranging from satellite antennas [12] to telescope lenses [1].

The ability of genetic algorithms (GAs) to develop unexpectedly novel solutions to design problems is best demonstrated with an early example from our research. A small lozenge-shaped robot with fixed body shape and fixed muscle placement was imported into our simulation environment, and muscle firing patterns evolved to produce locomotion. Rather than arriving at the anticipated forward-crawling motion along the long body axis, the GA instead discovered an effective sideways motion (as illustrated by the frames in Figure 1). (All movies referenced in this article can be seen on the corresponding author's Web page: http://cs.union.edu/∼rieffelj/videos.html.)

Figure 1. 

Genetic algorithms often arrive at unexpectedly novel results. When asked to develop a simple locomotive gait for this lozenge-shaped robot, the algorithm arrived at a sidewinding motion rather than the expected forward motion.

Figure 1. 

Genetic algorithms often arrive at unexpectedly novel results. When asked to develop a simple locomotive gait for this lozenge-shaped robot, the algorithm arrived at a sidewinding motion rather than the expected forward motion.

The research described in this article focuses on three evolutionary approaches to the problem of soft robot design and control. In each case we allow two properties to vary while the others remain fixed. (See Table 1.)

  • Gait-muscle: Given a fixed body shape, coevolving muscle placement and firing patterns.

  • Gait-material: Given a fixed morphology (shape and muscle placement), coevolving material properties alongside gaits.

  • Developmental: Given a fixed gait, evolving open-ended and variable morphologies (implicitly varying muscle placement and material properties).

Table 1. 

The three approaches.

Scheme
Body shape
Muscle placement
Material properties
Gait
Gait-muscle Fixed Evolved Fixed Evolved 
Gait-material Fixed Fixed Evolved Evolved 
Developmental Evolved Evolved Evolved Fixed 
Scheme
Body shape
Muscle placement
Material properties
Gait
Gait-muscle Fixed Evolved Fixed Evolved 
Gait-material Fixed Fixed Evolved Evolved 
Developmental Evolved Evolved Evolved Fixed 

In the case of the first inquiry (gait-muscle), we will show how properties like bilateral symmetry and oppositional muscles can emerge as the result of evolution. In case of the second approach (gait-material), we will demonstrate a sensitivity between material properties and gaits, and arrive at unexpected forms of locomotion. In the third approach (developmental), we will show how a developmental encoding can implicitly affect both material properties and muscle placement, and arrive at novel and surprising open-ended morphologies.

Salient to each approach is the problem of computational overhead imposed by simulating soft rather than rigid objects. Conventionally, soft-body simulators, ranging from finite element analysis (FEA) and computational fluid dynamics (CFD) to the off-the-shelf physics engine used in our research (NVidia's PhysX), rely upon tetrahedral meshes to represent soft bodies, and the complexity of simulation is directly tied to the number of tetrahedra in the mesh. For the fixed-morphology approach (gait-material), we introduce a method that scales model mesh resolution over the course of evolution, so that a large early portion of evolutionary time is devoted to low-resolution models of the robot, and as evolution progresses the mesh resolution increases. This resolution scaling achieves fitnesses comparable to those achieved by fixed high resolution while reducing overall computation time. For the evolved-morphology approach (developmental), we slowly increase the developmental lifetime of the soft robot over the course of evolution, thereby allowing the body to increase in complexity over time.

Each exploration not only provides insight into the creation of physically grounded soft robots, but provides feedback into fields such as biomedicine and biomechanics.

2 Simulating Soft Robots

Once the domain of FEA and CFD, physics simulation is now much more accessible, thanks to recent advances in commercial off-the-shelf video-game physics engines accelerated by massively parallel graphics cards (GPUs). This general-purpose computing on graphics processing units (GPGPU) can provide speedups of several orders of magnitude over software-only simulation [2]. In particular, our research uses NVidia's PhysX engine because of its ability to simulate complex three-dimensional soft bodies.

2.1 Representing Soft Bodies

In PhysX (as well as in FEA and CFD), soft bodies such as caterpillar robots are formed out of tetrahedral meshes. A single tetrahedron is defined by four vertices and four corresponding faces, as illustrated by Figure 2. A mesh is formed by connecting adjacent tetrahedra at common vertices.

Figure 2. 

A tetrahedron is defined by four vertices and four corresponding faces. The material properties of a mesh can betuned by changing the stretching and damping coefficients of spring-and-damper systems along the edge, and by changing the tetrahedron's resistance to volume changes.

Figure 2. 

A tetrahedron is defined by four vertices and four corresponding faces. The material properties of a mesh can betuned by changing the stretching and damping coefficients of spring-and-damper systems along the edge, and by changing the tetrahedron's resistance to volume changes.

2.1.1 Soft Body Material Properties

The material properties of a soft body mesh can be tuned by varying a set of constraints placed upon the tetrahedra within a mesh. Two parameters, the stretching stiffness and damping coefficient, tune the parameters of a spring-and-damper system along each edge of the tetrahedron. A tetrahedral mesh with high stretching stiffness will try hardest to maintain its shape, while one with a low stiffness will flop on the floor like a deflating balloon. The damping coefficient of a soft body determines how fast it returns to equilibrium after a perturbation. A low damping coefficient allows soft bodies in motion to oscillate more. A third constraint, volume stiffness, determines how hard each tetrahedron tries to maintain aconstant volume. Changing each of these values affects the softness of the all tetrahedra in a soft body, although not necessarily in a linear manner. As illustrated by Figure 3, by varying these material properties, the behavior of soft bodies in PhysX can be varied from a near fluid, to rubbery Jell-O, to a semi-rigid plastic. Finally, the friction of the crawling surface can be changed within a relatively narrow range in order to model how well the soft material grips the substrate.

Figure 3. 

Soft bodies in PhysX, such as the ones shown above, are built out of meshes of tetrahedra. Model resolution can vary with the number of tetrahedra in the mesh. Examples are shown with low mesh resolution (left) and high resolution (right). Changing the underlying material properties can drastically affect both the shape and the behavior of a soft body. Images are shown of the same soft body with high (right) and low (bottom) stretching stiffnesses.

Figure 3. 

Soft bodies in PhysX, such as the ones shown above, are built out of meshes of tetrahedra. Model resolution can vary with the number of tetrahedra in the mesh. Examples are shown with low mesh resolution (left) and high resolution (right). Changing the underlying material properties can drastically affect both the shape and the behavior of a soft body. Images are shown of the same soft body with high (right) and low (bottom) stretching stiffnesses.

The bottleneck for soft body simulation is the density of the underlying tetrahedral mesh: Simulation slows dramatically as the number of tetrahedra in a mesh grow (Figure 3). The tradeoff is that low-resolution meshes have lower fidelity to the real-world behavior of the corresponding physical soft body.

2.2 Soft Body Gaits

One of the more interesting consequences of soft robotics is the lack of conventional actuators. Because suppleness and deformability are important, devices like servos and stepper motors are not usable. Absent those, one valuable alternative is nitinol “memory wire” such as is used in Lin et al.'s GoQBot [11]. These artificial muscles act essentially as linear actuators, and can be modeled in PhysX by applying equal and opposite force vectors to two attachment vertices. Example layouts of the muscles on a typical soft robot are shown later in this article.

Once given a fixed set of muscles in a soft robot, their firing patterns can be modeled with a square wave characterized by a duty cycle, a phase offset, and a period (Figure 4). The period of the firing pattern is the time between the square wave's rising edges. The duty cycle is the percentage of time that a muscle ison during that period. Finally, the phase of the firing pattern is the delay before the first rising edge. Figure 5 shows typical bilateral symmetry in a robot.

Figure 4. 

Soft robot gaits are composed of firing patterns for each set of muscles. Firing patterns can be described by a duty cycle, phase, and period. Shown here are two gaits with different parameter values.

Figure 4. 

Soft robot gaits are composed of firing patterns for each set of muscles. Firing patterns can be described by a duty cycle, phase, and period. Shown here are two gaits with different parameter values.

Figure 5. 

An illustration of the linear actuator muscles of the simulated soft body. Although muscles are aligned with bilateral symmetry, no symmetry constraints were placed on the underlying firing patterns.

Figure 5. 

An illustration of the linear actuator muscles of the simulated soft body. Although muscles are aligned with bilateral symmetry, no symmetry constraints were placed on the underlying firing patterns.

3 Coevolving Muscle Placement and Gaits

One of the more fundamental choices in designing soft robots is in picking the placement of linear actuators. In our early design of soft robots [24] there were two assumptions that we took for granted, both influenced by biomimicry. The first is that muscle placement should be bilaterally symmetric, that is, that the left and right sides of the robot should be identical. The second is that, given the elastic nature of soft bodies, muscles acting in opposition were unnecessary—that the elastic prestress of the body wall would restore the body shape when the linear actuator was relaxed.

In order to challenge both of these assumptions, we chose to coevolve muscle placement alongside gait (muscle firing patterns) while keeping body shape and material properties fixed, with no constraint placed upon symmetry or opposition. The general scheme we used, as illustrated by Figure 6, was to have two parallel populations, one of gait genotypes and one of vertex genotypes. The merit in having a separate population and separate genotypes for gaits and vertices lies in balancing exploration and exploitation across the two population sets, much as in classical multi-armed-bandit problems.

Figure 6. 

Coevolving muscle placement and gaits involves two parallel populations, a population of attachment vertices and a population of muscle firing patterns. First, gaits are evolved using the current best attachment points; then, 200generations later, muscle attachment points are evolved using the current best gait.

Figure 6. 

Coevolving muscle placement and gaits involves two parallel populations, a population of attachment vertices and a population of muscle firing patterns. First, gaits are evolved using the current best attachment points; then, 200generations later, muscle attachment points are evolved using the current best gait.

The robot model we used was the caterpillar robot seen on the top in Figure 3 with 10 arbitrarily placed muscles. The firing pattern of a single muscle was encoded in a 3-tuple (period, phase, duty), and a genome for all 10 muscles encoded as 10 such 3-tuples. A single muscle's attachment vertices were represented as a pair (v1, v2), and a corresponding genome contained 10 such pairs. Each linearly encoded type of genome was subject to normal per-locus mutation and two-point crossover.

Evolution first progressed on the gait genotypes, using a hand-picked initial set of attachment vertices. After 200 generations, the current best gait was then used as a fixed reference with which to evolve the population of attachment points. After another 200-generation interval, the current best attachment points were used to evolve a new set of gaits. This seesaw pattern was repeated until 2,000 generations of each population had been evolved.

In each population, fitness was determined by the linear distance traveled by the soft body over a fixed number (8,000) of simulator time steps.

A representative solution is shown in Figure 7 (a video exists on the corresponding author's Web page). We can make two interesting qualitative observations about this evolved muscle placement. The first is that this solution, like most in our experiment, has a high degree of bilateral symmetry: Most muscles have a matching muscle on the opposite side.

Figure 7. 

Images of the left and right sides of a robot with evolved muscle placement and gaits (gait-muscle). The bottom image is mirrored to provide a more direct comparison (in both images the front of the robot is to the right.) When muscle placement is coevolved alongside gaits, bilateral symmetry emerges, even though it is not explicitly imposed. More unexpectedly, some muscles are placed in opposing pairs.

Figure 7. 

Images of the left and right sides of a robot with evolved muscle placement and gaits (gait-muscle). The bottom image is mirrored to provide a more direct comparison (in both images the front of the robot is to the right.) When muscle placement is coevolved alongside gaits, bilateral symmetry emerges, even though it is not explicitly imposed. More unexpectedly, some muscles are placed in opposing pairs.

The second, more surprising result is the emergence of muscles acting in opposition: In this case the middle leg on the lower image is pulled both forward and backward by a matched pair of muscles. While not strictly necessary, since the elastic force of the soft body will pull the leg back into place, an opposing muscle is able to move the leg forward more quickly and thereby speed up the gait.

While largely qualitative, these results show how relaxing otherwise assumed design constraints allows genetic algorithms to both confirm those assumptions (with respect to symmetry) and improve solutions by contradicting them (with respect to opposition).

4 Coevolving Gaits and Material

The next question we ask is whether, given a specific body plan with specific muscle placements, we can coevolve firing patterns alongside material properties. It is worth emphasizing that this question is grounded in real-world soft robotics applications: many physical soft robots employ silicone elastomers, whose material properties can be changed quite significantly during the mixing process [25]8.

Figure 8. 

Similarly, coevolving gaits with physical properties requires a population of gaits and a population of physical properties.

Figure 8. 

Similarly, coevolving gaits with physical properties requires a population of gaits and a population of physical properties.

Our goal in this case was to simultaneously discover a suitably matched gait-property pair. Because ofthe dynamics of a soft body in motion, the fitness of a specific gait can vary greatly depending upon theunderlying material properties, and, similarly, the fitness of a material property set depends greatly upon the gait it is tested against. Our coevolutionary scheme in this case (Figure 8) is identical to that in Section 3, except that while one population similarly contained gait genomes for eight (rather than 10) muscles, the other contained genomes representing material properties (as described in Section 2.1.1) that were limited to keep results realistic. Ranges are as shown in Table 2 (note that in PhysX, as in most physics simulators, these properties are unitless).

Table 2. 

Property values were limited to specific ranges to promote realistic behavior.

Property
Min
Max
Volume stiffness 0.1 1.0 
Stretching stiffness 0.3 1.0 
Friction 0.1 1.0 
Damping 0.0 1.0 
Property
Min
Max
Volume stiffness 0.1 1.0 
Stretching stiffness 0.3 1.0 
Friction 0.1 1.0 
Damping 0.0 1.0 

Initially, a fixed best guess of material property values was used for evaluating the fitness of each gait. The second population evolved soft body material properties, where a single genome contained values for a specific set of stretching stiffness, volume stiffness, damping coefficient, and body friction. Initially, for this population's fitness evaluations, a fixed best guess of firing patterns was used.

Every tenth generation, the gait used for material property fitness evaluation was updated with the current highest-fitness gait from the gait population, and the material properties used for gait population evaluation were updated from the highest-fitness property values.

As a measure of elapsed wall time, with population size 40, a typical run took 24 h to evaluate 100 generations of each population on a 2.66-GHz Core i7 processor with 6 GB of RAM.

We have two analyses of our experiments to offer. The first is a more qualitative description of the gaits produced by our system and some insight into how changing material properties affect fitness. The second more quantitatively explores the effect of scaling the mesh resolution over the course of an evolutionary run.

4.1 Analyzing Effects of Material Properties on Gaits

Our experiments consistently produced interesting and effective gaits, and analysis suggests that being able to change material properties alongside firing patterns has a positive effect upon the outcome. A qualitative and visual representation lies in videos of the actual gaits available on our Web page.

One video shows a bipedal—that is, bilaterally asymmetric—gait. Firing patterns on each side of the body coordinate in a rough front-to-back wave pattern in order to collectively lift the limbs upward and forward during the upswing, before relaxing into the downswing to pull against the ground. The relative softness of the material can be seen in the amount of flexing undergone by each leg. A second video, by comparison, shows a more symmetrical gait achieved by a forward-moving wave, which produces what looks almost like a gallop.

There were some distinct differences in material property values across these two runs, as summarized below (see Table 3).

Table 3. 

Material property values.

Property
Bipedal
Wave
Volume stiffness 0.986 0.996 
Stretching stiffness 0.982 0.998 
Friction 0.598 0.804 
Damping 0.0004 0.0 
Property
Bipedal
Wave
Volume stiffness 0.986 0.996 
Stretching stiffness 0.982 0.998 
Friction 0.598 0.804 
Damping 0.0004 0.0 

The most notable difference is the friction—corresponding to the stickiness of the robot's feet. However, when watching the videos, the relatively minor numerical differences in the other property values appear, at least qualitatively, to be reflected in the behavior of the soft bodies. Figure 9 shows the comparison between a static mesh and a single resolution switch.

Figure 9. 

A comparison of coevolutionary progress under a static mesh resolution (bottom) with that under a single resolution switch. The sudden drop in fitness corresponding to resolution change is caused by the low-resolution gait working lesswell on the higher-resolution mesh.

Figure 9. 

A comparison of coevolutionary progress under a static mesh resolution (bottom) with that under a single resolution switch. The sudden drop in fitness corresponding to resolution change is caused by the low-resolution gait working lesswell on the higher-resolution mesh.

Of further interest are the changes in best values of material properties that occur over the course of an evolutionary run, as shown in Figure 10. While the changes appear small, they comprise a significant range of the allowable values, as described by Table 2. While damping coefficient and volume stiffness show nearly monotonic progress toward a fixed value, stretching stiffness and friction vary consistently across a relatively wide range during evolution. The correlation between material property changes and fitness is even more apparent when shown alongside the corresponding fitness graph (shown at the bottom of Figure 10). For instance, the large swing in damping coefficient at generation 7 corresponds to a significant rise in fitness. Other, smaller fitness gains also appear to have corresponding material value changes.

Figure 10. 

Material property values for population bests per generation. Large swings in values, such as the drop in damping at generation 7, are associated with large improvements in fitness (bottom graph).

Figure 10. 

Material property values for population bests per generation. Large swings in values, such as the drop in damping at generation 7, are associated with large improvements in fitness (bottom graph).

4.2 Scaling Mesh Resolution

Our second analysis is of the benefits offered by scaling mesh resolution over the course of evolution. Recall that the number of tetrahedra in a mesh is the determining factor in simulation running time, as well as in simulator fidelity. Our intuition was that the bulk of early evolutionary time, which largely consists of the soft robot flailing around (i.e., attempting to achieve nonzero fitness), could be performed on relatively low meshes, and then, as evolution and fitness progressed, mesh resolution could be scaled up to raise the emphasis on fidelity at the cost of longer evaluation times. Our hypothesis was that this switching would allow for faster evolutionary progress (when measured by elapsed wall time rather than number of generations).

We created five hard-coded mesh resolutions (Low, MedLow, Med, High, and Max) by running meshing software on the preexisting 3D caterpillar robot model. We then performed experiments allowing 0, 1, 2, or 4 mesh resolution increases at 30-generation intervals over the course of evolution. All of the non-static runs shown began on the Low mesh—runs that are listed with a mesh switch count of 1, for instance, changed from the Low mesh to their end mesh. Runs listed with a mesh switch count greater than 1 ran on an intermediate mesh(es) before reaching their end mesh. All other properties, such as population size, remained constant across experiments. The results of this suite are summarized in Table 4.

Table 4. 

A summary of results from resolution scaling. The outcomes of the first set of boldface experiments are shown in Figure 9, and those of the second set are shown in Figure 11.

Fitness
Hours
End mesh
Mesh switches
40.16 34 Low 
46.94 42 Low 
24.61 72 Max 
 
14.86 25 MedLow 
21.47 25 MedLow 
 
15.99 48 Med 
16.65 48 Med 
15.91 48 Med 
 
6.18 46 Max 
Fitness
Hours
End mesh
Mesh switches
40.16 34 Low 
46.94 42 Low 
24.61 72 Max 
 
14.86 25 MedLow 
21.47 25 MedLow 
 
15.99 48 Med 
16.65 48 Med 
15.91 48 Med 
 
6.18 46 Max 

Figure 9, which compares evolution with a single (Low-to-MedLow) switch to that with a fixed (MedLow) mesh resolution, shows the consequences of this process: Fitness in the scalable evolution during its Low phase progresses much more rapidly during the first 10 h of simulation. Once the phase change to a higher mesh density occurs, however, there is a dramatic drop in fitness, and the scalable run loses much of the ground it had gained (though it still remains above the fixed-resolution result). During the following 15 h, the scalable run is able to make up much of the lost fitness, and improves more rapidly than the static mesh.

This steep loss in fitness is a result of the large extent to which the success of a gait is highly tuned to its specific mesh resolution. The same actually holds true of the evolving material properties as well. Gaits and physical properties evolved at one mesh resolution simply do not translate perfectly when placed in a higher-resolution simulation.

This dependence on mesh resolution also has a clear effect upon the maximum obtainable evolutionary fitness: Over similar time scales, even the static meshes show significant differences in final fitness.

The last entry in Table 4 illustrates the cost of switching clearly: The final fitness is less than half of that achieved by any other run. This suggests that, in its current form, sometimes the cost of resolution scaling can be too high. Figure 11 shows a case where even a single switch in resolution results in an equivocal, at best, improvement in overall fitness.

Figure 11. 

Results comparing the consequences of switching mesh resolution multiple times over the course of evolution. Each time the resolution changes, there is a dramatic drop in fitness due to the relatively poor translation of gaits and material properties into the higher mesh.

Figure 11. 

Results comparing the consequences of switching mesh resolution multiple times over the course of evolution. Each time the resolution changes, there is a dramatic drop in fitness due to the relatively poor translation of gaits and material properties into the higher mesh.

The source of this loss in fitness can possibly be illustrated with an interesting qualitative distinction of gaits evolved at varying resolutions: All the gaits evolved in a low-resolution mesh produced bipedal gaits, whereas gaits produced in the Max mesh tended to be more bilaterally symmetric, involving instead a forward-propagating wavelike motion. In other words, a high-fitness bipedal gait evolved at a low mesh resolution ceases to be competitive when placed in a higher-resolution body.

Mesh scaling certainly holds promise and, in a few cases illustrated above, offers an improvement over static-resolution evolution, despite the large fitness drops associated with resolution switches. While it remains to be seen if this is a viable way to address the issue of long simulation times, we are hopeful of itsprospects. Our next section explores an alternative method for variable mesh resolution.

5 Evolving Body Shape Development

In our final approach we use a grammatically based developmental encoding to evolve complete soft robot shapes in an open-ended manner. As we will show, this allows us to address several of the challenges of soft robotics discussed above.

Developmental encodings have been used with considerable success to create the morphology of simulated robotics [4, 5, 9, 10, 20], most often in conjunction with off-the-shelf physics engines. While there are several different flavors of generative encoding (among them L-systems [10, 16], genetic regulatory networks [3], and HyperNEAT [22]—Stanley and Miikkulainen [23] provide a useful taxonomy), they are largely used to create rigid structures, and none explicitly operates on tetrahedral meshes.

For our purposes, the open-ended generation of tetrahedral meshes, we have created a face-encodingL-system [17] capable of growing multi-resolution tetrahedral-mesh robot morphologies. L-systems use a sequence of rewrite rules that operate on the faces of tetrahedra. Similar encodings operating on graph edges rather than tetrahedral faces have been used to grow both 3D surfaces [6] and tensegrity structures [18].

The details of our particular encoding are as follows. Assuming that each face of a tetrahedron can be given a label, we specify three operations which can be performed upon a face, as illustrated in Figure 12. For the sake of simplicity, we assert that these operators can only be applied to exposed faces—that is, those that are not shared by two adjacent tetrahedra—most commonly, but not exclusively, on the exposed outer surface of a tetrahedral mesh. A rule set can then be created by specifying a fixed number of nonterminal labels and providing a rewrite rule for each label. A detailed description of this encoding is provided by an earlier article of ours [17].

Figure 12. 

An illustration of the three rules that can be applied to the face of a tetrahedron. Clockwise from top left: the original tetrahedron with face labeled A; relabel replaces A with B, subdivide replaces the face with four smaller faces (this requires subdividing the entire tetrahedron), and grow adds a new tetrahedron with face labels B, C, D.

Figure 12. 

An illustration of the three rules that can be applied to the face of a tetrahedron. Clockwise from top left: the original tetrahedron with face labeled A; relabel replaces A with B, subdivide replaces the face with four smaller faces (this requires subdividing the entire tetrahedron), and grow adds a new tetrahedron with face labels B, C, D.

Tetrahedral meshes of arbitrary size can now be grown by iteratively applying a rule set to an initial seed tetrahedron like the one shown in Figure 2. Each exposed face of the growing tetrahedral mesh is kept in a queue, and is associated with three vertices and exactly one tetrahedron. (A face shared by two tetrahedra is by definition not exposed.) For every generation of growth, the open faces are iteratively removed from the queue and the appropriate rule is applied. For relabel, a new face with the new label is enqueued. For grow and divide, new vertices and tetrahedra are computed and added, and then the resulting three (grow) or four (subdivided) new faces are enqueued. Figure 13 shows the growth of one such tetrahedral mesh.

Figure 13. 

The growth of a tetrahedral mesh on iteratively applying the rules from a face-encoding rule set.

Figure 13. 

The growth of a tetrahedral mesh on iteratively applying the rules from a face-encoding rule set.

5.1 Implicitly Variable Mesh Resolution

A valuable consequence of this encoding, particularly in light of our efforts in Section 4.2, is that it can result in tetrahedral meshes of variableresolution: Each subdivide operation on a face converts the single parent tetrahedron into eight smaller tetrahedra, any of which can in turn be subdivided into eight smaller tetrahedra 1/64 the size of the original tetrahedron. Meanwhile, any tetrahedron not subject to face subdivisions will retain its original dimensions. Since the time complexity of a soft body simulation is determined largely by the total number of tetrahedra, this variable resolution allows for a certain reduction in simulation time compared to an identical tetrahedral mesh composed of homogeneously sized tetrahedra. As we will see in the next section, this has consequences for differential stiffness within the mesh as well.

Rule sets describe a developmental process, and thus they are an indirect encoding of a tetrahedral mesh physical phenotype. These grammars can be evolved by treating the rule sets as genotypes and the tetrahedral mesh that results after a fixed number of iterations as the phenotype. With a grammar as a genotype, an evolving population simply consists of a collection of these grammar genotypes.

5.2 Moving without Linear Actuators

Using linear actuators with attachment vertices as in our previous two approaches would have further complicated the grammar. We chose instead to periodically vary the stiffness of the tetrahedral mesh, which results in corresponding deformations in the soft body itself. (Hiller and Lipson used a similar approach in their amorphous robots [7], which they were able to implement physically using closed cell foam and a vacuum chamber [8].) These changes in stiffness can be considered in many ways analogous to the state change of muscles between passive and active forms, in which stiffness significantly increases with activation [13]. It is worth noting that in this PhysX implementation a uniform change in stiffness can result in nonuniform deformations, since a region of smaller tetrahedra will deform more than a region with a single large tetrahedron when under similar loads, largely due to the increased number of vertices around which the tetrahedra can flex.

5.3 Evolving Morphology with Grammars

Rule sets such as the one shown in Table 5 describe a developmental process, or ontogeny. Thus, they are an indirect encoding of a physical phenotype. These grammars can be evolved by treating the rule sets as genotypes, and the tetrahedral mesh that results after a fixed number of iterations as the phenotype. Rule sets can be encoded as simple linear strings of characters subject to mutation and crossover. Mutation on a grammar genotype affects only the right-hand side of a rule, and can either change the rule (e.g., grow(A, B, D) → relabel(A), with extra labels added or removed as necessary) or change a label in the rule (e.g., grow(A, B, D) → grow(E, B, D). Single-point crossover grabs a subset of production rules from one parent, and the remainder from a second. An evolving population simply consists of a collection of grammar genotypes.

Table 5. 

An example rule set.

A → grow {DBF
B → grow {ADF
C → grow {EDF
D → relabel (D
E → grow {DCF
F → divide [DDDG
G → grow {DDG
A → grow {DBF
B → grow {ADF
C → grow {EDF
D → relabel (D
E → grow {DCF
F → divide [DDDG
G → grow {DDG

For the experiments that produced the results described below, we used a fixed population size of20, and 50% elitism. Parents were chosen with a simple fitness-proportional selection. 40% ofoffspring were produced via crossover, and the remainder via mutation. An edit-distance diversitymetric was employed to prevent multiple neutral mutations of a single genotype schema from proliferating.

Each individual was evaluated by applying its genotype grammar rewrite rules a fixed number of times (20, 40, or 100) in order to produce a phenotype, and then placing the resulting robot in the PhysX environment. The tetrahedral mesh stiffness was then cycled between between maximal stiffness and 80% of maximal (again, all parameters in PhysX are unitless), with a period of 200 time steps, and the displacement vector over each cycle was recorded. The total distance traveled during a 4,000-time-step window was then computed by adding the displacement vectors.

In order to prevent the common but pathological trait of toppling, in which evolving bodies move their centers of mass by simply falling over (first made famous by Sims' evolved agents [21]), the displacement vectors were passed through a low-pass filter, which effectively ignores short-term high velocities associated with toppling, while preserving the steady state long-term lower velocities due to actual locomotion.

In each experiment, evolution proceeded for 500 generations. Each grammar was fixed at seven labels A through G, with seven corresponding rewrite rules.

5.4 Results and Discussion

Every experiment across all iteration levels (20, 40, 100) produced forward motion of the soft bodies, with no fitness exploitation via toppling. Videos of several of the evolved soft robots can be seen on our Web page. Figure 14 compares fitness over time across the three iteration levels. As might be expected, the bodies produced with 100 iterations traveled further in the same amount of time than those generated with either 40 or 20 iterations.

Figure 14. 

A comparison of evolutionary progress of tetrahedral meshes produced after 20 iterations against those produced after 40 or 100 iterations. More iterations tend to produce larger meshes.

Figure 14. 

A comparison of evolutionary progress of tetrahedral meshes produced after 20 iterations against those produced after 40 or 100 iterations. More iterations tend to produce larger meshes.

Figure 15 contains frames from one such video. In almost every case, the evolved gait involves a“scooting” or ratcheting motion, where cyclic flexing on one end robot pushes the robot in the opposite direction.

Figure 15. 

Frames from a video of the evolved locomotion. Cyclic changes in stiffness affect the leg on the left, and ratchets the rest of the body to the right.

Figure 15. 

Frames from a video of the evolved locomotion. Cyclic changes in stiffness affect the leg on the left, and ratchets the rest of the body to the right.

Elements of body symmetry and modularity are visible in most of the evolved morphology. Symmetry is somewhat limited by the choice to hard-code the faces of the seed tetrahedron as ABCD—this meant that in order to achieve two radially symmetric limbs growing from the original tetrahedron, rewrite rules that relabeled two of the starting faces to a third common face would be required, something along the lines of ArelabelG, BrelabelG. In future experiments we intend to make the initial starting labels a component of the genome, which will allow for more symmetry, and also for more large-scale coordinated changes in the phenotype.

5.4.1 Grammars Implicitly Specify Both Muscle Function and Material Properties

As illustrated by the results, the multi-resolution nature of the evolved tetrahedral meshes allows fora significant degree of regional functional differentiation. What we mean by this is that regions of the body composed of larger tetrahedra (lower resolution) tend to remain stiff over the course of locomotion—thereby providing structure to the body—whereas regions with many smaller tetrahedra flex more, and thereby act more like muscles. Figure 16 illustrates this phenomenon more clearly. The regions of smaller tetrahedra on the left of each frame compress and buckle as their stiffness is reduced, while the larger tetrahedra pivot around the buckling. As the stiffness increases again, the structure is propelled to the right.

Figure 16. 

An illustration of the regional functional differentiation exhibited by the evolved meshes. A region with smaller tetrahedra, indicated by the arrow, is more flexible and tends to buckle as the stiffness is decreased, while larger tetrahedra are less affected.

Figure 16. 

An illustration of the regional functional differentiation exhibited by the evolved meshes. A region with smaller tetrahedra, indicated by the arrow, is more flexible and tends to buckle as the stiffness is decreased, while larger tetrahedra are less affected.

While the grammar explicitly determines body shape, this regional differentiation is an emergent property of our grammar. This is quite valuable in that it allows our grammars to implicitly specify both material properties (in that regions with smaller tetrahedra are more flexible) and function (regions with smaller tetrahedra act like muscles). Moreover, since mesh resolution can vary heterogeneously throughout the body, we are allowed a much finer-grained control over mesh resolution than the explicit resolution switching used in Section 4.

5.5 An Avenue toward Open-Ended Greater Scalability

It is worth noting that Figure 14 is slightly misleading in that this is not a purely objective comparison. Tetrahedral meshes generated with 20 iterations generally have fewer tetrahedra than those generated with 40 or 100 iterations. Since computational complexity is proportional to the number of tetrahedra, a comparison of the evolutions over the three parameter values normalized for elapsed wall time might be better. We look forward to pursuing such a comparison in future work.

This does, however, lead to a second observation, more quantitative in nature, which can be seen when the developmental trajectory of an evolved solution is inspected more closely. Over the course of evolution, the fitness of the tetrahedral mesh is measured only once—after a predetermined number of rewrite rules have been applied. There is, however, an underlying ontogenic process, which means that every intermediate phenotype is itself a complete and possibly viable body shape. Of course, since in our grammatical encoding there are no limits on body shape or size other than the iteration limit, there is no true sense of “fully grown”—there is no feedback in the system that might give rise to a more fixed or stable phenotype. Nonetheless, given an evolved grammar, we can measure the fitness of every intermediate stage of development.

Since the fitness after N rewrites is the only one that matters in our evolutionary setup, the fitnesses of prior (and subsequent) developmental stages are not selected for, and therefore there is no expectation that those stages would have any significant fitness. However, as the graphs in Figure 17 illustrate, instead there can be a high degree of fitness throughout the developmental trajectory. Sometimes, as in the top graph in Figure 17, there is a sharp jump in fitness near the fixed 20-iteration limit, but the fitness remains quite stable for an additional 10 iterations before slowly decaying. In other cases (middle graph), fitness builds gradually (and nonmonotonically) up to the iteration limit, then drops to around 30% of nominal fitness for the next 10 iterations, and actually jumps up above 50% of nominal fitness after 15 iterations. In a third case (bottom graph), there are ontogenic stages before and after the iteration limit that are twice as high as the evolved fitness value.

Figure 17. 

Phenotype fitness measured over the course of development, and normalized around the fitness measured at the fixed iteration limit (vertical line). Although during evolution the fitness is only measured at that iteration limit, it often remains stable as development proceeds past the limit (top graph), and is sometimes higher at earlier and later developmental stages.

Figure 17. 

Phenotype fitness measured over the course of development, and normalized around the fitness measured at the fixed iteration limit (vertical line). Although during evolution the fitness is only measured at that iteration limit, it often remains stable as development proceeds past the limit (top graph), and is sometimes higher at earlier and later developmental stages.

This suggests a new avenue to follow for this developmental encoding: gradually scaling the number of iterations over the course of evolution. This would in effect offer a scaling process very similar to that we describe in Section 4, but without the need to hand-code mesh resolution. Our hope is that such an approach will allow for greater scalability both in body complexity and in reduced computational load. We are similarly interested in a closer analysis of the emergence of modularity and substructure repetition in developing phenotypes.

5.6 Tetrahedral Robots Can Be Printed

Although these tetrahedral meshes were evolved in simulation, they are exceptionally easy to fabricate inthe real world. A stereo lithography (STL) file, readable by all modern 3D printers and rapid prototyping machines, can be generated from a tetrahedral mesh description simply by enumerating the open faces. Figure 18 provides an example of one such mesh, which was printed out ofABS plastic on a Stratasys printer. Printed robots need not be rigid, of course: State-of-the-artprinters, such as those designed by Objet, are now able to print in much softer materials, and even cheap desktop prototypers like the Makerbot Cupcake can print 3D shapes out of silicone elastomers.

Figure 18. 

Tetrahedral meshes (left) can easily be converted into stereo lithography format and printed with high fidelity on a 3D printer (right).

Figure 18. 

Tetrahedral meshes (left) can easily be converted into stereo lithography format and printed with high fidelity on a 3D printer (right).

6 Conclusion: Three Ways to Evolve Soft Robots

The evolution of completely soft robots is a many-headed problem. The three central challenges—morphology, material, and control—are all interdependent, and a solution to any one is predicated upon existing solutions to the other two.

We have explored three ways to approach this problem, in each case holding one property constant while coevolving the other two. In the first two cases, each with fixed morphology, this required two separately evolving populations, where the current best from one parameter's population (e.g., gait) was used as the baseline to evolve a second parameter (e.g., muscle location). In the third case, we held control constant while using a developmental encoding, which was able to simultaneously control both muscle placement and material properties by varying the tetrahedral mesh size.

While each approach highlights the challenges involved in soft robot evolution, each leads to interesting insights as well.

The results from our first approach, gait-muscle, suggest that bilateral symmetry and oppositional muscles can be valuable in soft robotic gaits. Our second approach, gait-material, highlights the tight coupling between soft robotic material properties and gaits, and suggests that both should be evolved in tandem. This emphasis on the effects of body dynamics has had a recent biological analog as well, in the discovery that in the Manduca sexta caterpillar, the biomimetic muse of this research, the internal gut plays a surprisingly important role in locomotion [19].

We find the most promise in our third approach, developmental, through which a generative encoding is able to explicitly generate body shape while implicitly determining heterogeneous material properties throughout the body as well as muscle placement. In our upcoming work we look forward to leveraging the developmental stability of the grammatical encoding in order to find a way to scale to increasingly large body sizes without determining the number of iterations a priori.

Our results are a valuable illustration of the coevolution of morphology and control. Taken collectively, we hope that these insights can inform our efforts at achieving physically embodied soft robots. While practical applications are a long way off, success in this regard will have valuable applications infields ranging from soft and flexible urban search and rescue robots to more compliant biomedical and orthoscopic devices.

Acknowledgments

This work was supported in part by the US Defense Advanced Research Project Agency under Grant W911SR08-C-0012 (ChemBots) and by the US National Science Foundation IOS-7045912 to Barry Trimmer. Undergraduate research by Davis Knox was facilitated by Union College's Undergraduate Research Program.

References

1. 
Al-Sakran
,
S. H.
,
Koza
,
J. R.
, &
Jones
,
L. W.
(
2005
).
Automated re-invention of a previously patented optical lens system using genetic programming.
In M. Keijzer, A. Tettamanzi, P. Collet, J. I. van Hemert, & M. Tomassini (Eds.)
,
Proceedings of the 8th European Conference on Genetic Programming
(pp.
25
37
).
Berlin
:
Springer
.
2. 
Banzhaf
,
W.
, &
Harding
,
S.
(
2009
).
Accelerating evolutionary computation with graphics processing units.
In
GECCO '09: Proceedings of the 11th Annual Conference Companion on Genetic and Evolutionary Computation
(pp.
3237
3286
).
New York
:
ACM
.
3. 
Bongard
,
J.
(
2002
).
Evolving modular genetic regulatory networks.
In
Proceedings of the 2002 Congress on Evolutionary Computation. CEC '02
,
Vol. 2
(pp.
1872
1877
.
4. 
Bongard
,
J.
, &
Pfeifer
,
R.
(
2003
).
Morpho-functional machines: The new species (designing embodied intelligence)
(pp.
237
258
).
Berlin
:
Springer-Verlag
.
5. 
Bongard
,
J. C.
, &
Pfeifer
,
R.
(
2001
).
Repeated structure and dissociation of genotypic and phenotypic complexity in artificial ontogeny.
In L. Spector et al (Eds.)
,
Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2001)
(pp.
829
836
).
San Mateo, CA
:
Morgan Kaufmann
.
6. 
Hemberg
,
M.
, &
O'Reilly
,
U.-M.
(
2004
).
Extending grammatical evolution to evolve digital surfaces with genr8
. In
EuroGP.
7. 
Hiller
,
J. D.
, &
Lipson
,
H.
(
2010
).
Evolving amorphous robots.
In H. Fellermann, M. Dörr, M. M. Hanczyc, L. L. Laursen, S. Maurer, D. Merkle, P.-A. Monnard, K. Stoy, & S. Rasmussen (Eds.)
,
Artificial Life XII: Proceedings of the Twelfth International Conference on the Simulation and Synthesis of Living Systems
(pp.
717
724
).
Cambridge, MA
:
MIT Press
.
8. 
Hiller
,
J. D.
, &
Lipson
,
H.
(
2010
).
Morphological evolution of freeform robots.
In
Proceedings of the 12th Annual Conference on Genetic and Evolutionary Computation, GECCO '10
(pp.
151
152
).
New York
:
ACM
.
9. 
Hornby
,
G. S.
, &
Pollack
,
J. B.
(
2001
).
The advantages of generative grammatical encodings for physical design.
In
Proceedings of the 2001 Congress on Evolutionary Computation, CEC2001
(pp.
600
607
).
Piscataway, NJ
:
IEEE Press
.
10. 
Hornby
,
G. S.
, &
Pollack
,
J. B.
(
2001
).
Evolving L-system to generate virtual creatures.
Computers and Graphics
,
25
(
6
),
1041
1048
.
11. 
Lin
,
H.-T.
,
Leisk
,
G. G.
, &
Trimmer
,
B.
(
2011
).
GoQBot: A caterpillar-inspired soft-bodied rolling robot.
Bioinspiration & Biomimetics
,
6
(
2
),
026007
.
12. 
Lohn
,
J. D.
,
Hornby
,
G. S.
, &
Linden
,
D. S.
(
2005
).
An evolved antenna for deployment on NASA's Space Technology 5 mission.
In U.-M. O'Reilly, R. L. Riolo, T. Yu, & B. Worzel (Eds.)
,
Genetic programming theory and practice II.
Dordrecht, The Netherlands
:
Kluwer
.
13. 
Paetsch
,
C.
,
Trimmer
,
B.
, &
Dorfmann
,
A.
(
2012
).
A constitutive model for active-passive transition of muscle fibers.
International Journal of Non-Linear Mechanics
,
47
,
377
387
.
14. 
Pollack
,
J.
,
Lipson
,
H.
,
Funes
,
P.
,
Ficici
,
S.
, &
Hornby
,
G.
(
1999
).
Coevolutionary robotics.
In
EH '99: Proceedings of the 1st NASA/DOD Workshop on Evolvable Hardware
(p.
208
).
IEEE Computer Society
.
15. 
Pollack
,
J. B.
,
Lipson
,
H.
,
Hornby
,
G.
, &
Funes
,
P.
(
2001
).
Three generations of automatically designed robots.
Artificial Life
,
7
(
3
),
215
223
.
16. 
Prusinkiewicz
,
P.
, &
Lindenmayer
,
A.
(
1990
).
The algorithmic beauty of plants.
New York
:
Springer-Verlag
.
17. 
Rieffel
,
J.
, &
Smith
,
S.
(
2010
).
A face-encoding grammar for the generation of tetrahedral-mesh soft bodies.
In H. Fellermann et al (Eds.)
,
Artificial Life XII: Proceedings of the Twelfth International Conference on the Simulation and Synthesis of Living Systems
(pp.
414
420
).
Cambridge, MA
:
MIT Press
.
18. 
Rieffel
,
J.
,
Valero-Cuevas
,
F.
, &
Lipson
,
H.
(
2009
).
Automated discovery and optimization of large irregular tensegrity structures.
Computers & Structures
,
87
(
5–6
),
368
379
.
19. 
Simon
,
M.
,
Woods
,
W.
,
Serebrenik
,
Y.
,
Simon
,
S.
,
Griethuijsen
,
L. V.
,
Socha
,
J.
,
Lee
,
W.
, &
Trimmer
,
B.
(
2010
).
Visceral-locomotory pistoning in crawling caterpillars.
Current Biology
,
20
,
1458
1463
.
20. 
Sims
,
K.
(
1994
).
Evolving 3D morphology and behavior by competition.
In R. Brooks & P. Maes (Eds.)
,
Artificial Life IV, Proceedings
(pp.
28
39
).
Cambridge, MA
:
MIT Press
.
21. 
Sims
,
K.
(
1994
).
Evolving virtual creatures.
In
Proceedings of the 21st Annual Conference on Computer Graphics and Interactive Techniques
(pp.
15
22
).
New York
:
ACM Press
.
22. 
Stanley
,
K. O.
,
D'Ambrosio
,
D. B.
, &
Gauci
,
J.
(
2009
).
A hypercube-based encoding for evolving large-scale neural networks.
Artificial Life
,
15
,
185
212
.
23. 
Stanley
,
K. O.
, &
Miikkulainen
,
R.
(
2002
).
A taxonomy for artificial embryogeny.
Artificial Life
,
9
(
2
),
93
130
.
24. 
Trimmer
,
B. A.
(
2008
).
New challenges in biorobotics: Incorporating soft tissue into control systems.
Applied Bionics and Biomechanics
,
5
(
3
),
119
126
.
25. 
Trimmer
,
B. A.
,
Lin
,
H.-T.
,
Leisk
,
G.
, &
Kaplan
,
D.
(
2012
).
Towards a biomorphic soft robot: Design constraints and solutions.
In
Proceedings of the Fourth IEEE RAS/EMBS International Conference on Biomedical Robotics and Biomechatronics
(pp.
599
605
).

Author notes

Contact author.

∗∗

Union College, Schenectady, NY 12308. E-mail: rieffelj@union.edu (J.R.)

Tufts University, Medford, MA 02155.