Parameterized runtime analysis seeks to understand the influence of problem structure on algorithmic runtime. In this paper, we contribute to the theoretical understanding of evolutionary algorithms and carry out a parameterized analysis of evolutionary algorithms for the Euclidean traveling salesperson problem (Euclidean TSP). We investigate the structural properties in TSP instances that influence the optimization process of evolutionary algorithms and use this information to bound their runtime. We analyze the runtime in dependence of the number of inner points k. In the first part of the paper, we study a EA in a strictly black box setting and show that it can solve the Euclidean TSP in expected time where A is a function of the minimum angle between any three points. Based on insights provided by the analysis, we improve this upper bound by introducing a mixed mutation strategy that incorporates both 2-opt moves and permutation jumps. This strategy improves the upper bound to . In the second part of the paper, we use the information gained in the analysis to incorporate domain knowledge to design two fixed-parameter tractable (FPT) evolutionary algorithms for the planar Euclidean TSP. We first develop a EA based on an analysis by M. Theile, 2009, ”Exact solutions to the traveling salesperson problem by a population-based evolutionary algorithm,” Lecture notes in computer science, Vol. 5482 (pp. 145–155), that solves the TSP with k inner points in generations with probability . We then design a EA that incorporates a dynamic programming step into the fitness evaluation. We prove that a variant of this evolutionary algorithm using 2-opt mutation solves the problem after steps in expectation with a cost of for each fitness evaluation.
In many real applications, the inputs of an NP-hard combinatorial optimization problem may be structured or restricted in such a way that it becomes tractable to solve in practice despite having a worst-case exponential time bound. Parameterized analysis seeks to address this by expressing algorithmic runtime in terms of an additional hardness parameter that isolates the source of exponential complexity in the problem structure. In this paper, we study the application of evolutionary algorithms (EAs) to the Euclidean traveling salesperson problem (TSP) and consider the runtime of such algorithms as a function of both problem size and a further parameter that influences how hard the problem is to solve by an EA.
1.1 The Euclidean Traveling Salesperson Problem
Iterative heuristic methods (such as local search and evolutionary algorithms) that rely on the exchange of a few edges such as the well-known 2-opt (or 2-change) operator are popular choices for solving large-scale TSP instances in practice. This is partly due to the fact that they have a simple implementation and typically perform well empirically. However, for these algorithms, theoretical understanding still remains limited. Worst-case analyses demonstrate the existence of instances on which the procedure can be inefficient. Chandra et al. (1999), building on unpublished results due to Lueker, have shown that local search algorithms employing a k-change neighborhood operator can take exponential time to find a locally optimal solution. Even in the Euclidean case, Englert et al. (2007) have recently shown that a local search algorithm employing inversions can take worst-case exponential time to find tours that are locally optimal.
If the search operator is restricted to specialized 2-opt moves that remove only edges that intersect in the plane, van Leeuwen and Schoone (1981) proved that a tour that has no such planar intersections can be reached in moves, even if a move introduces further intersecting edges. If the vertices are distributed uniformly at random in the unit square, Chandra et al. (1999) showed that the expected time to find a locally optimal solution is bounded by . More generally, for so-called -perturbed Euclidean instances, Englert et al. (2007) proved that the expected time to find a locally optimum solution is bounded by . These results also imply similar bounds for simple ant colony optimization algorithms as shown in Kötzing et al. (2012).
To allow for a deeper insight into the relationship between problem instance structure and algorithmic runtime, we appeal in this paper to the theory of parameterized complexity (Downey and Fellows, 1999; Flum and Grohe, 2006). Rather than expressing the runtime solely as a function of problem size, parameterized analysis decomposes the runtime into further parameters that are related to the structure of instances. The idea is to find parameters that partition off the combinatorial explosion that leads to exponential runtimes (Downey and Fellows, 1999).
In the context of TSP, a number of parameterized results currently exist. Deneko et al. (2006) showed that, if a Euclidean TSP instance with n vertices has k vertices interior to the convex hull, there is a dynamic programming algorithm that can solve the instance in time bounded by , where g is a function that depends only on k. This means that this parameterization belongs to the complexity class FPT, the class of parameterized problems that are considered fixed-parameter tractable. Of course, membership in FPT depends strongly on the parameterization itself. For example, the problem of searching the k-change neighborhood for metric TSP is hard for due to Marx (2008). , but the containment is conjectured to be proper (Flum and Grohe, 2006).
1.2 Computational Complexity of Evolutionary Algorithms
Initial studies on the computational complexity of EAs consider their runtime on classes of artificial pseudo-Boolean functions (Droste et al., 2002; He and Yao, 2001, 2003; Yu and Zhou, 2006). The goal of these studies is to consider the impact of the different modules of an EA and to develop new methods for their analysis. This early work was instrumental in establishing a rigorous understanding of the behavior of EAs on simple functions, for identifying some classes of problems that simple EAs can provably solve in expected polynomial time (Droste et al., 2002), and for disproving widely accepted conjectures (e.g., that evolutionary algorithms are always efficient on unimodal functions; Droste et al., 1998).
More recently, classical polynomial-time problems from combinatorial optimization such as minimum spanning trees (Neumann and Wegener, 2006, 2007) and shortest paths (Baswana et al., 2009; Doerr et al., 2011; Scharnow et al., 2004) have been considered. In this case, one does not hope to beat the best problem-specific algorithms for classical polynomial solvable problems. Instead, these studies provide interesting insights into the search behavior of these algorithms and show that many classical problems are solved by general-purpose algorithms such as EAs in expected polynomial time.
Research on NP-hard combinatorial optimization problems such as makespan scheduling, covering problems, and multi-objective minimum spanning trees (Neumann, 2007; Witt, 2005) show that EAs can achieve good approximations for these problems in expected polynomial time. In the case of the TSP, Theile (2009) has proved that a EA based on dynamic programming can exactly solve the TSP in at most steps when is allowed to be exponential in n. For a comprehensive presentation of the different results that have been achieved see, for example, the recent text of Neumann and Witt (2010).
Algorithmic runtime on NP-hard problems can be studied in much sharper detail from the perspective of parameterized analysis, and this has only recently been started in theoretical work on EAs. Parameterized results have been obtained for the vertex cover problem (Kratsch and Neumann, 2009), the problem of computing a spanning tree with a maximal number of leaves (Kratsch et al., 2010), variants of maximum 2-satisfiability (Sutton et al., 2012), and makespan scheduling (Sutton and Neumann, 2012).
1.3 Our Results
In this paper, we carry out a parameterized complexity analysis for evolutionary algorithms on the planar Euclidean TSP. Specifically, using the parameterization of Deneko et al. (2006), we prove upper bounds on the expected time for certain EAs to solve planar TSP instances as a function of n, the number of points in the instance, and of k, the number of inner points, that is, the points that lie on the interior of the convex hull of the planar TSP instance. Our motivation is to use this parameterization from the classical algorithms community to understand the influence of the number of inner points on the runtime of a black box EA, and then to design EAs that exploit this problem structure to obtain fixed-parameter tractable runtimes.
In the first part of the paper, we investigate EAs applied to the planar Euclidean TSP entirely from a black box perspective. We consider the parameterized runtime bounds on algorithms that do not have access to any concrete problem information. Specifically, in the black box scenario, algorithms can only access the fitness value of each candidate solution as measured by tour length. Our results are for the EA which operates on a population of permutations (candidate Hamiltonian cycles) and produces offspring in each generation using a mutation operator based on 2-opt. This analysis provides further insight into the optimization process that allows us to design a mixed mutation operator that uses both 2-opt moves and permutation jumps and improves the upper bound on the expected runtime of the EA. This part of the paper is an extension of work presented at AAAI 2012 (Sutton and Neumann, 2012).
In the second part of the paper, we leave the black box setting and consider EAs that have access to both fitness values and the ordering of the points that lie on the convex hull. This ordering can be precomputed in polynomial time. Using this approach, we design two fixed-parameter tractable (FPT) evolutionary algorithms for the parameterization of Deneko et al. This first algorithm builds on the previous study of Theile (2009) and the general framework for evolutionary algorithms for dynamic programming presented in Doerr et al. (2011). It works with a large population where each individual can be seen as an entry of the dynamic programming table of the classical approach. The population size grows exponentially with the number of inner points and we assume that this result is mainly of theoretical interest.
While the first FPT algorithm is interesting mostly from a theoretical perspective, our second FPT algorithm is highly practical as a heuristic search algorithm. Instead of searching for a permutation of the n given points, we apply evolutionary search only to find a good permutation of the k inner points. The outer points (lying on the convex hull) have to appear in cycling order according to the convex hull in any optimal solution. For a fixed permutation x on the inner points, the best possible tour according to and x can be computed in time using dynamic programming. We make use of these ideas and propose to use EAs that work with permutations of the inner points. Our theoretical investigations show that commonly used mutation operators for permutations such as inversions, jumps, and exchanges yield fixed-parameter EAs. Furthermore, our experimental investigations confirm that this approach leads to significantly better results if the number of inner points is not too large. This part of the paper is an extension of work presented at CEC 2013 (Nallaperuma et al., 2013).
This paper is organized as follows. In Section 2 we introduce the problem, state some preparatory material, and discuss parameterized analysis. In Section 3 we study the runtime of a EA on Euclidean TSP instances in a black box setting. We first consider the time it takes for such algorithms to solve instances whose points lie in convex position, that is, have no inner points. We then prove rigorous runtime bounds for the algorithms as a function of the number of inner points in an instance. In Section 4, we design two EAs with access to some instance-specific information and prove that they are fixed-parameter tractable EAs, that is, they solve the planar TSP with k inner points in expected time where g depends only on k. We conclude the paper in Section 5.
Geometrically, it will often be convenient to consider an edge as the unique planar line segment with end points u and v. We say a pair of edges and intersect if they cross at a point in the Euclidean plane. An important observation, which we state here without proof, is that any pair of intersecting edges form the diagonals of a convex quadrilateral in the plane (see Figure 1).
If and intersect at a point p, they form the diagonals of a convex quadrilateral described by points u, s, v, and t. Hence edges , , , and form a set of edges that mutually do not intersect.
A tour is called intersection-free if it contains no pairs of edges that intersect.
The convex hull of V is the smallest convex set containing V. A point is called an inner point if v lies in the interior of the convex hull of V. We denote as the set of inner points of V. A point is called an outer point. The set of such points we denote by . Hence and partition V.
Note that every tour for all permutations x on V corresponds to a set of edges that describe a closed polygon in the plane. The covertex class of a polygon is the class of all polygons that have the same vertex set as the given polygon. The following two theorems are due to Quintas and Supnick (1965).
A planar polygon that is shortest of its covertex class and whose vertices are noncollinear cannot intersect itself.
A planar polygon that is shortest of its covertex class and whose vertices are noncollinear must contain the vertices on the boundary of its convex hull in their cyclic order.
We call the unique cyclic order that arises from the points in hull-order. We say a permutation x respects hull-order if any two points in the subsequence of x induced by are consecutive in x only if they are adjacent along the boundary of the convex hull of V. Thus, Theorem 2 states that, if x is a minimum of f, then x respects hull-order. We will actually need a slightly stronger result than the one provided by the claim of Theorem 2. This is captured by the following.
If is intersection-free, then x respects hull-order.
Lemma 1 follows from arguments in the proof of Theorem 2 above. In particular, Quintas and Supnick (1965) show that if a polygon does not contain the vertices on the boundary of the convex hull in their cyclic order, then the polygon must intersect itself (and hence, by Theorem 1, cannot be optimal). The claim of the lemma follows then by contraposition.
2.1 Parameterized Analysis
Parameterized complexity theory is an extension to traditional computational complexity theory in which the analysis of hard algorithmic problems is decomposed into parameters of the problem input. This approach illuminates the relationship between hardness and different aspects of problem structure because it often isolates the source of exponential complexity in NP-hard problems.
A parameterization of a problem is a mapping of problem instances into the set of natural numbers. We are interested in expressing algorithmic complexity in terms of both problem size and the extra parameter. Formally, let L be a language over a finite alphabet . A parameterization of L is a mapping . The corresponding parameterized problem is the pair .
For a string , let and . An algorithm deciding in time bounded by is called a fixed-parameter tractable (or fpt-) algorithm for the parameterization . Here is an arbitrary but computable function. Similarly, an algorithm that decides a parameterized problem in time bounded by is called an -algorithm.
When working with the runtime of randomized algorithms such as evolutionary algorithms, one is often interested in the random variable T which somehow measures the number of generations the algorithm must take to decide a parameterized problem. A randomized algorithm with expected optimization time (respectively, ) is a randomized fpt-algorithm (respectively, -algorithm) for the corresponding parameterization .
In the case of the Euclidean TSP on a set of points V, we want to express the runtime complexity as a function of n and k where and k is the number of inner points: the vertices that lie in the interior of the convex hull of V. For the corresponding optimization problem, we are interested in the runtime until the optimal solution is located.
3 Black Box EAs
We begin by investigating the parameterized runtime behavior of simple randomized search heuristics on the Euclidean TSP purely from a black box perspective. From this perspective, we assume that the heuristics do not have any access to problem information other than the fitness of an individual as measured by the tour length induced by the corresponding permutation.
3.1 Permutation Operators
Randomized search heuristics that are applied to solving TSP instances are usually designed to iteratively search the space of permutations in order to minimize the fitness function defined in Equation (1). Each permutation corresponds to a particular Hamiltonian cycle in the graph. To move through the space of candidate solutions, move or mutation operators are often constructed based on some kind of elementary operations on the set of permutations on V. For the black box algorithms, we will consider two such operations: inversions and jumps, which we define as follows.
In the space of Hamiltonian cycles, the permutation inversion operation is essentially identical to the well-known 2-opt (or 2-change) operation for TSP. The usual effect of the inversion operation is to delete the two edges and from and reconnect the tour using edges and (see Figure 2). Here and subsequently, we consider arithmetic on the indices to be modulo n, that is, and . Since the underlying graph G is undirected, when , the operation has no effect because the current tour is only reversed. There is also no effect when . In this case, it is straightforward to check that the edges removed from are equal to the edges replaced to create .
Since and have the same effect, there are unique jump operations. The jump operator was used by Scharnow et al. (2004) in the context of runtime analysis of EAs on permutation sorting problems.
3.2 The EA
Let be the stochastic process that is a restriction of constructed by taking only permutations in the same order. We consider the drift of the random sequence .
Again, since for any there is a nonzero probability the algorithm exits the state x and transits to the absorbing state . This event is characterized as an independent Bernoulli trial with success probability at least q. It follows that the series in the claim can be estimated by a geometrically distributed random variable with parameter q, and therefore the expected time spent in states contained in the partition is bounded above by .
In Section 3.3 we will introduce a constraint on the angles between edges that give rise to suitable values for c in the claim of Lemma 2. This, along with corresponding values for p and q (that appear in the claims of Lemmas 2 and 3, respectively), will allow us to bound the expected runtime in terms of n and k. Before doing so, it will be convenient to state a number of lemmas that will be useful in the analysis. To preserve the flow of the text, we defer their proofs to the appendix.
Let x be a permutation such that is not intersection-free. Then there exists an inversion that removes a pair of intersecting edges and replaces them with a pair of non-intersecting edges.
Note that it is still possible that the introduced edges intersect with some of the remaining edges in .
Suppose . Then there are at most distinct intersection-free tours.
Suppose and is an intersection-free tour on V. Then there is a sequence of at most k jump operations that transforms x into an optimal permutation. Similarly, there is a sequence of at most inversion operations that transform x into an optimal permutation.
3.3 Angle-Bounding, Quantization, and Resulting Structural Properties
A challenge to the runtime analysis of algorithms that employ edge exchange operations such as 2-opt is that, when points are allowed in arbitrary positions, the minimum change in fitness between neighboring solutions can be made arbitrarily small. Indeed, proof techniques for worst-case analysis often leverage this fact (Englert et al., 2007). To circumvent this, we impose bounds on the angles between points, which allows us to express runtime results as a function of trigonometric expressions involving these bounds. Momentarily, we will refine this further by introducing a class of TSP instances embedded in an grid. In that case, we will see that the resulting trigonometric expression is bounded by a polynomial in m.
We say V is angle-bounded by for some if for any three points , where denotes the angle formed by the line from u to v and the line from v to w. This allows us to express a bound in terms of on the change in fitness from a move that removes an inversion.
The lower bound on the angle between any three points in V provides a constraint on how small the change in fitness between neighboring inversions can be. This lower bound is useful in the case of a quantized point set. This occurs when the points can be embedded on an grid as illustrated in Figure 5.
Quantization, for example, occurs when the x and y coordinates of each point in the set are rounded to the nearest value in a set of m equidistant values (e.g., integers). We point out that it is still important that the quantization preserve the constraint on collinearity since collinear points violate a nonzero angle bound. We have the following lemma.
Suppose V is a set of points that lie on an unit grid, no three collinear. Then V is angle-bounded by .
Lemma 8 allows us to translate the somewhat awkward trigonometric expression in the claim of Lemma 7 (and subsequent lemmas that depend on it) into a convenient polynomial that can be expressed in terms of m.
Let V be a set of n points that lie on an unit grid, no three collinear. Then, V is angle-bounded by where .
We are now ready to prove the following technical lemma for 2-opt mutation (see Algorithm 3). This lemma will be instrumental in proving runtime bounds later in the paper.
Let V be a set of planar points in convex position angle-bounded by . We have the following.
1. For any , the probability that 2-opt mutation creates an offspring y with is at least .
2. For any , the probability that 2-opt mutation creates an optimal solution is at least
For (1), suppose . By Lemma 4, there is at least one pair of intersecting edges in that can be removed with a single inversion operation . Let . By Lemma 7, satisfies the fitness bound in the claim. It suffices to bound the probability that y is produced by 2-opt-mutation (x).
Let E1 denote the event that Poisson mutation performs exactly one inversion (i.e., s = 0 in line 2 of Algorithm 3). Let E2 denote the event that the pair is specifically chosen for the inversion.
For (2), suppose . Thus, is intersection-free, and it follows from Lemma 6 that there are at most inversion moves that transform x into an optimal solution.
Let denote the event that Poisson mutation performs exactly inversions (i.e., in line 2 of Algorithm 3). Let denote the event that all inversions are the correct moves that transform x into an optimal solution.
Finally, since the time bounds in the remainder of this paper are expressed as a function of the angle bound , and the bounds on point distance, it will be useful to define the following function.
3.4 Instances in Convex Position
A finite point set V is in convex position when every point in V is a vertex on its convex hull. Deneko et al. (2006) observed that the Euclidean TSP is easy to solve when V is in convex position. In this case, the optimal permutation is any linear ordering of the points which respects the ordering of the points on the convex hull. Such an ordering can be found in time (de Berg et al., 2008).
In the context of EAs the natural question arises, if V is in convex position, how easy is it for simple randomized search heuristics? In this case, a tour is intersection-free if and only if it is globally optimal, hence finding an optimal solution is exactly as hard as finding an intersection-free tour. Since an intersection can be (at least temporarily) removed from a tour by an inversion operation (cf. Lemma 4), we focus in this section on algorithms that use the inversion operation.
We begin our analysis of instances in convex position by considering RLS. RLS operates on a single candidate solution by performing a single inversion in each iteration. Since each inversion which removes an intersection results in a permutation whose fitness is improved by the amount bounded in Lemma 7, it is now straightforward to bound the time it takes for RLS to discover a permutation that corresponds to an intersection-free tour.
Let V be a set of planar points in convex position angle-bounded by . The expected time for RLS to solve the TSP on V is where A is defined in Definition 6.
If V is in convex position and embedded in an grid with no three points collinear, then RLS solves the TSP on V in expected time .
The bound follows immediately from Theorem 3 since V is in convex position and since, by Lemma 8, V is angle-bounded by , , and . Appealing to Lemma 9 yields . Substituting these terms into the bound of Theorem 3 completes the proof.
We now consider the optimization time of the EA using 2-opt mutation defined in Algorithm 1 applied to a set of points in convex position. Obviously, in this case we will find and terms appearing in the runtime formulas. We will assume that and are polynomials in both n and k. We will also find that setting ensures that a transition from any state to a state which improves on the fitness by at least a specified amount occurs with constant probability.
Such a setting has the effect of reducing the number of generations spent removing intersections from the best-so-far tours. However, it is important to note that the number of calls to the fitness function must be accordingly increased by a factor of in each generation. Nevertheless, the expected number of generations can be a useful measure when considering parallel EAs. As Jansen et al. (2005) have pointed out, when the fitness evaluation of the offspring can be performed on parallel processors, the number of generations corresponds to the parallel optimization time. In such a case, we would observe a quadratic factor improvement in the parallel runtime corresponding to the segment spent in tours with intersections. Nevertheless, we also report the bound on the number of total fitness function evaluations.
The following theorem bounds the number of expected generations that the EA needs to solve the Euclidean TSP on a set of angle-bounded points in convex position.
Let V be a set of planar points in convex position angle-bounded by . The expected time for the EA using 2-opt mutation to solve the TSP on V is bounded above by where A is as defined in Definition 6.
Case 1:. For this setting of , we have , so an intersection is removed in each generation with constant probability. Invoking Lemma 2, the expected time to find an intersection-free tour is at most .
Thus, the expected time to solve the instance is bounded by which yields the claimed bound.
We immediately get the following corollary.
Let V be a set of planar points in convex position angle-bounded by . The expected number of fitness evaluations needed by the EA using 2-opt mutation to solve the TSP on V is bounded above by , and the expected optimization time of the EA is bounded above by , where A is as defined in Definition 6.
Moreover, analogous to the corollary to Theorem 3, we have the following.
If V is in convex position and embedded in an grid with no three points collinear, then the EA solves the TSP on V in expected time
3.5 Instances with k Inner Points
We now turn our attention to TSP instances in which and express the expected runtime in terms of , the number of points, and , the number of inner points.
3.5.1 2-opt Mutation
We use the structural analysis in Section 3.3 to show that when there are few inner points, intersection-free tours are somehow “close” to an optimal solution in the sense that relatively small perturbations by the EA suffice to solve the problem. Again, recall that we assume and are polynomials in both n and k.
Let V be a set of points angle-bounded by such that . The expected time for the EA using 2-opt mutation to solve the TSP on V is bounded above by .
We argue by analyzing the Markov chain generated by the EA using 2-opt mutation. Let denote the sequence of best-so-far states visited by the EA.
We also get the following corollary.
Let V be a set of points angle-bounded by such that . The expected number of fitness evaluations needed for the EA using 2-opt mutation to solve the TSP on V is bounded above by and the expected optimization time for the (1 + 1) EA is .
Again, from Lemmas 8 and 9 we also have the following corollary.
Let V be a set of points quantized on an grid such that . The expected time for the EA using 2-opt mutation to solve the TSP on V is .
3.5.2 Mixed Mutation Strategies
Similar to Lemma 10, we have the following result for mixed mutation.
Let V be a set of planar points in convex position angle-bounded by . Then,
1. For any , the probability that mixed mutation creates an offspring y with is at least .
2. For any , the probability that mixed mutation creates an optimal solution is at least .
For (1), the probability that mixed mutation selects inversions is and the rest of the claim follows from the argument of Lemma 10.
For (2), suppose . Thus, is intersection-free, and it follows from Lemma 6 that there are at most k jump operations that transform x into an optimal solution. The remainder of the proof is identical to that of Lemma 10. The probability that mixed mutation selects jump operations contributes the leading factor of .
Let V be a set of points angle-bounded by such that . The expected time for the EA using mixed mutation to solve the TSP on V is bounded above by .
The proof is identical to the proof of Theorem 5, except that we substitute the probabilities from Lemma 11 into Lemmas 2 and 3.
We immediately get the following corollary.
Let V be a set of points angle-bounded by such that . The expected number of fitness evaluations needed for the EA using mixed mutation to solve the TSP on V is bounded above by and the expected optimization time for the (1 + 1) EA is bounded above by .
As before, from Lemmas 8 and 9 we have the following.
Let V be a set of points quantized on an grid such that . The expected time for the EA using mixed mutation to solve the TSP on V is .
4 Fixed-Parameter Evolutionary Algorithms
Moving away from the black box perspective, we now consider designing EAs that are given access to a small amount of problem information. Specifically, we allow the algorithms access to the ordering of points that lie on the convex hull. Using this information, we design two fixed-parameter tractable EAs for the inner point parameterization of Deneko et al. (2006).
4.1 A Population-Based Approach
In this section we describe a EA that solves the TSP to optimality in fixed-parameter tractable time. We build on the evolutionary dynamic programming framework for TSP introduced by Theile (2009). We show that a slight modification to Theile’s EA that carefully maintains the invariant that the points in remain in convex-hull order for each individual allows us to improve the time bound significantly on instances for which the number of inner points is not too large.
3. x is -respecting, that is, appear in order.
For a permutation , we call the set its ground set and we call r its tail vertex.
The TSP tour defined by a permutation is a cycle , that is, it starts at p1, runs over all nodes in such that any nodes in are visited in the order they appear in , and finally visits r before returning to p1. The full population of the EA consists of an individual for every , every , and possible tail vertex r.
In order to introduce sufficiently large changes to the population in each generation, we utilize the Poisson mutation strategy in the same manner as both Theile’s original algorithm and the black box EAs introduced in Section 3. Specifically, in each generation, for each of the selected individuals, we apply the mutation operator defined above times, where s is drawn from a Poisson distribution with unit expectation.
Finally, we again employ truncation selection in the sense that each mutated offspring may only replace the individual in the parent population with the same ground set and tail vertex. This replacement occurs only if the fitness of the mutated offspring is at least as good as the parent individual in question. Thus, at the end of a generation, the parent population size is preserved.
We now bound the expected time until the EA described in Algorithm 6 has solved a TSP instance on n points with k inner points to optimality. We first bound the population size with the following lemma.
The parent population size is bounded above by .
The population is initialized by constructing, for each possible distinct ground set and each legal tail vertex r a length permutation such that , , and the elements from appear in the same order as they do in . Note that to maintain the correct ordering, the tail vertex cannot be any point pj for , or the ordering with respect to is violated. Therefore, the tail vertex can be any member of .
After the population P is initialized, for each ground set and each legal tail vertex , there exists some permutation that respects the order .
The proof of the runtime is a straightforward extension of Theile's (2009) argument that the EA can simulate dynamic programming on a function similar to F in a randomized fashion.
Let V be a set of n points in the Euclidean plane with . After generations, the EA has solved the TSP on V to optimality in expectation and with probability .
Suppose there exists an individual such that . Since selection for reproduction occurs uniformly at random, with probability at least , x is selected for reproduction to produce one of the offspring by mutation in each subsequent generation. Furthermore, with probability , only a single mutation step is performed on x (i.e., following line 11 of Algorithm 6). Finally, the probability that x is extended in an optimal fashion is at least , that is, the probability that the correct vertex q is chosen out of to create an optimal subpath on the ground set .
Case 2:. For this setting of we have and the number of generations until the optimal solution is found is at most .
To prove the concentration result, for some constant and any , let be the number of generations needed to produce an optimal subpath of length m where . We define a sequence of random indicator variables for all such that if the correct mutation has occurred to extend an optimal subpath in the ith generation. Let . The expectation of X is , by the above bounds on .
The expected number of fitness evaluations is .
4.2 Searching for Permutations of Inner Points
In this section, we propose an approach that makes use of insights in the context of parameterized complexity. By Theorem 2, the outer points are always visited in hull-order in every optimal solution. In the following, we assume that is the fixed order on the outer points given in Definition 7. Let x be a permutation on the inner points . Using the dynamic programming algorithm of Deneko et al., one can compute the shortest Hamiltonian cycle respecting and x in time .
These ideas suggest that using a heuristic to only search for an optimal permutation on the inner points can significantly reduce the effective size of the search space, and thus lead to quicker solution times for EAs. For each permutation on the inner points x, the quality can be evaluated by running a dynamic programming algorithm to compute the best possible tour using and x.
Let be the fixed order of the outer points and be an arbitrary permutation of the inner points. We can compute the shortest tour respecting and x using the dynamic programming approach of Deneko et al. (2006), which we state in the following.
The dynamic programming algorithm computes a three-dimensional table F with entries where , , and . The algorithm computes for each triple the cost of the shortest path from p1 through all nodes and respecting the orders and x. The path ends at pi iff determines that the last node is an outer point and ends at qj iff determines that the last node is an inner point.
The time computing the F array is proportional to its size which is . Therefore, the fitness for a given permutation x of the inner points can be computed in time .
For mutation, we employ the inversion (2-opt) and permutation jump mutation operators introduced in Section 3.1. In the case of inversions, however, it is no longer clear whether this is the correct choice for the new search space, which is restricted only to orderings on the inner points. In addition to inversions and jumps (see Definitions 2 and 3), we also consider the exchange operator, which exchanges two elements (chosen uniformly at random) in the ordering of a permutation. We formalize the exchange operator in the following definition.
We denote by () EA, () EA, and () EA, the variants of the () EA using inversions, jumps, and exchanges, respectively.
In the following, we investigate the number of operations needed in order to obtain an optimal solution in dependence of the chosen mutation operator. This will lead to different upper bounds on the runtime of the algorithm. For all the operators, we show that they yield fixed-parameter tractable results.
Let V be a set of n points in the Euclidean plane with . The expected number of generations needed to solve the TSP on V for both the () EA, and the () EA is bounded by . For the () EA, the bound is .
Let be a lower bound on the probability to transform an arbitrary non-optimal solution into an optimal solution. Suppose there are no optimal solutions in the population. The probability that at least one of the offspring is optimal is bounded below by .
As usual, the expected number of fitness evaluations can be derived by where is bounded by Theorem 8.
The previous theorem shows that simple EAs are fixed-parameter tractable EAs for the considered problem when working with permutations on the inner points and applying dynamic programming as part of the fitness evaluation. Note that dynamic programming takes time . This is more expensive than computing the cost of a given tour on n cities, which can be done in time. However, it is still feasible to use the dynamic programming approach as part of each fitness evaluation. In the next section, we show that this also leads to a significantly better optimization process than working with the standard approach that searches through the set of permutations on all n cities.
We consider several planar Euclidean TSP instances of various sizes (25, 50, and 100) and, within each size class, control the percentages of inner points. The instance generation process is described in detail in the next section. We calculate the solution quality reached within a given time limit on each instance. On these instances, we run the EAs using each mutation strategy for both the (1 + 1) EA and the (1 + 1) EA. As a baseline, we also compute the value of the optimal solution on each instance using the exact TSP solver Concorde (Applegate et al., n.d.).
For each run, we set time limits based on the instance size. Hence, we run each EA for 1 s on instances with 25 points, 3 s for instances with 50 points, and finally 15 s for instances with 100 points. We measure the iteration count which is the number of fitness function calls for each EA. The (1 + 1) EA runs roughly 200,000 iterations within the given time limits for all instance sizes. On the other hand, the (1 + 1) EA performs in the range of 1000–5000 iterations. The number of iterations performed by the (1 + 1) EA decreases as a function of the ratio of outer points to inner points in the instance. The experiments are conducted on a PC with 2.4 GHz Intel Core i5 CPU and 4GB 1,333 MHz DDR3 RAM.
4.3.1 Parameterized Instances
Since we need to study the impact of the number of inner points on algorithm performance, we cannot use standard benchmark instances, since it is not clear how to cleanly control the number of inner points in such problem sets. Instead, we generate random instances controlling for the percentage of points that lie in the interior of the convex hull. For each size category , we generate a set of 30 instances each with inner point percentages 5%, 10%, 20%, 30%, and 40%. For example, the set contains 30 TSP instances on planar points such that exactly lie in the interior of the convex hull. To construct such instances, we first generate the outer points on a convex hull, and then add the required number of inner points. We create a circle with a given radius and randomly generate outer points on the circumference by producing a random angle and then deriving x and y coordinates from the angle with respect to the y axis. This process continues until we have constructed the required values for . We then identify the convex hull and generate random points inside the hull within a distance of zero to the radius. Here, we accept a randomly generated inner point only if it falls inside the convex hull.
We measured the solution quality obtained within the allocated time for the six (1 + 1) EA variants described in this paper. We also compared each with the optimal solution computed by Concorde. The dataset consists of 30 instances from each group representing each instance size and each inner node percentage.
The results suggest that the (1 + 1) EA approach consistently leads to more effective optimization when there are not too many inner points. In Table 1, we see the average values of solution quality obtained for the instances in each instance size and inner point percentage. For some cases, variants of (1 + 1) EA (e.g., the set of with 5% inner points) reach the optimal values computed by the exact solver Concorde, while in most instances, they optimize the instances more effectively than the corresponding variants that search through permutations on all n points ((1 + 1) EA). In a few cases, where the inner point percentage is high, the new approach tends to perform poorly. This is expected, since the construction procedures are leveraging the constraints on the convex hull. Therefore, we expect the performance to degrade as the number of inner points becomes large with respect to the problem size.
|Instance .||Exchanges .||Inversions .||Jumps .||.|
|n/% inner .||(1 + 1) EA .||(1 + 1) EA .||(1 + 1) EA .||(1 + 1) EA .||(1 + 1) EA .||(1 + 1) EA .||Concorde .|
|Instance .||Exchanges .||Inversions .||Jumps .||.|
|n/% inner .||(1 + 1) EA .||(1 + 1) EA .||(1 + 1) EA .||(1 + 1) EA .||(1 + 1) EA .||(1 + 1) EA .||Concorde .|
To further investigate the effect of mutation strategy, we examine the three variants of the (1 + 1) EA for the problem set and 40% inner points. The solution quality reached in the allocated time (15 s) is plotted as a function of instance for each variant in Figure 6.
To determine whether the (1 + 1) EA outperforms the (1 + 1) EA with statistical significance, we perform a one-tailed Wilcoxon signed rank test (Wilcoxon, 1945) on the solution quality measurements obtained after the allocated runtime. The resulting positive rank sum (W) and confidence (p values) are recorded. We perform the test for each mutation strategy on each instance, and observe the effect of inner point percentage and size. To perform these analyses, we use the GNU R software environment (R Development CoreTeam, 2011). The results are summarized in Table 2 where the control variables (instance size and inner point percentage) are indicated in the first column.
|Instance .||Exchanges .||Inversions .||Jumps .|
|n/% inner .||W .||p .||W .||p .||W .||p .|
|Instance .||Exchanges .||Inversions .||Jumps .|
|n/% inner .||W .||p .||W .||p .||W .||p .|