We extend previous results concerning black box search algorithms, presenting new theoretical tools related to no free lunch (NFL) where functions are restricted to some benchmark (that need not be permutation closed), algorithms are restricted to some collection (that need not be permutation closed) or limited to some number of steps, or the performance measure is given. Minimax distinctions are considered from a geometric perspective, and basic results on performance matching are also presented.

A genetic algorithm is invariant with respect to a set of representations if it runs the same no matter which of the representations is used. We formalize this concept mathematically, showing that the representations generate a group that acts upon the search space. Invariant genetic operators are those that commute with this group action. We then consider the problem of characterizing crossover and mutation operators that have such invariance properties. In the case where the corresponding group action acts transitively on the search space, we provide a complete characterization, including high-level representation-independent algorithms implementing these operators.

In a previous paper (Rowe et al., 2002), aspects of the theory of genetic algorithms were generalised to the case where the search space, Ω, had an arbitrary group action defined on it. Conditions under which genetic operators respect certain subsets of Ω were identified, leading to a generalisation of the term schema . In this paper, search space groups with more detailed structure are examined. We define the class of structural crossover operators that respect certain schemata in these groups, which leads to a generalised schema theorem. Recent results concerning the Fourier (or Walsh) transform are generalised. In particular, it is shown that the matrix group representing Ω can be simultaneously diagonalised if and only if Ω is Abelian. Some results concerning structural crossover and mutation are given for this case.

It is supposed that the finite search space Ω has certain symmetries that can be described in terms of a group of permutations acting upon it. If crossover and mutation respect these symmetries, then these operators can be described in terms of a mixing matrix and a group of permutation matrices. Conditions under which certain subsets of Ω are invariant under crossover are investigated, leading to a generalization of the term schema . Finally, it is sometimes possible for the group acting on Ω to induce a group structure on Ω itself.

Holland's schema theorem (an inequality) may be viewed as an attempt to understand genetic search in terms of a coarse graining of the state space. Stephens and Waelbroeck developed that perspective, sharpening the schema theorem to an equality. Of particular interest is a “form invariance” of their equations; the form is unchanged by the degree of coarse graining. This paper establishes a similar form invariance for the more general model of Vose et al. and uses the attendant machinery as a springboard for an interpretation and discussion of implicit parallelism.

This paper continues the development, begun in Part I, of the relationship between the simple genetic algorithm and the Walsh transform. The mixing scheme (comprised of crossover and mutation) is essentially “triangularized” when expressed in terms of the Walsh basis. This leads to a formulation of the inverse of the expected next generation operator. The fixed points of the mixing scheme are also determined, and a formula is obtained giving the fixed point corresponding to any starting population. Geiringer's theorem follows from these results in the special case corresponding to zero mutation.

This paper is the first part of a two-part series. It proves a number of direct relationships between the Fourier transform and the simple genetic algorithm. (For a binary representation, the Walsh transform is the Fourier transform.) The results are of a theoretical nature and are based on the analysis of mutation and crossover. The Fourier transform of the mixing matrix is shown to be sparse. An explicit formula is given for the spectrum of the differential of the mixing transformation. By using the Fourier representation and the fast Fourier transform, one generation of the infinite population simple genetic algorithm can be computed in time O ( c l log 2 3 ), where c is arity of the alphabet and l is the string length. This is in contrast to the time of O ( c 3 l ) for the algorithm as represented in the standard basis. There are two orthogonal decompositions of population space that are invariant under mixing. The sequel to this paper will apply the basic theoretical results obtained here to inverse problems and asymptotic behavior.

A complete generalization of the Vose genetic algorithm model from the binary to higher cardinality case is provided. Boolean AND and EXCLUSIVE-OR operators are replaced by multiplication and addition over rings of integers. Walsh matrices are generalized with finite Fourier transforms for higher cardinality usage. Comparison of results to the binary case are provided.

This paper speaks to the inherent emergent behavior of genetic search. For completeness and generality, a class of stochastic search algorithms, random heuristic search , is reviewed. A general convergence theorem for this class is then proved. Since the simple genetic algorithm (GA) is an instance of random heuristic search, a corollary is a result concerning GAs and time to convergence.

The infinite- and finite-population models of the simple genetic algorithm are extended and unified, The result incorporates both transient and asymptotic GA behavior. This leads to an interpretation of genetic search that partially explains population trajectories. In particular, the asymptotic behavior of the large-population simple genetic algorithm is analyzed.

The infinite population simple genetic algorithm is a discrete dynamical system model of a genetic algorithm. It is conjectured that trajectories in the model always converge to fixed points. This paper shows that an arbitrarily small perturbation of the fitness will result in a model with a finite number of fixed points. Moreover, every sufficiently small perturbation of fimess preserves the finiteness of the fixed point set. These results allow proofs and constructions that require finiteness of the fixed point set. For example, applying the stable manifold theorem to a fixed point requires the hyperbolicity of the differential of the transition map of the genetic algorithm, which requires (among other things) that the fixed point be isolated.

A general form of stochastic search is described ( random heuristic search ), and some of its general properties are proved. This provides a framework in which the simple genetic algorithm (SGA) is a special case. The framework is used to illuminate relationships between seemingly different probabilistic perspectives of SGA behavior. Next, the SGA is formalized as an instance of random heuristic search. The formalization then used to show expected population fitness is a Lyapunov function in the infinite population model when mutation is zero and fitness is linear. In particular, the infinite population algorithm must converge, and average population fitness increases from one generation to the next. The consequence for a finite population SGA is that the expected population fitness increases from one generation to the next. Moreover, the only stable fixed point of the expected next population operator corresponds to the population consisting entirely of the optimal string. This result is then extended by way of a perturbation argument to allow nonzero mutation.

The original schema theorem (an inequality) has been replaced by an equality that determines the expected next generation for a simple genetic algorithm. This has made possible the computation of the trajectory of expected next generations. Visualization of these evolutionary trajectories beginning from different initial populations has led to the discovery of fractal structures.