## Abstract

Parameterized runtime analysis seeks to understand the influence of problem structure on algorithmic runtime. In this paper, we contribute to the theoretical understanding of evolutionary algorithms and carry out a parameterized analysis of evolutionary algorithms for the Euclidean traveling salesperson problem (Euclidean TSP). We investigate the structural properties in TSP instances that influence the optimization process of evolutionary algorithms and use this information to bound their runtime. We analyze the runtime in dependence of the number of inner points *k*. In the first part of the paper, we study a EA in a strictly black box setting and show that it can solve the Euclidean TSP in expected time where *A* is a function of the minimum angle between any three points. Based on insights provided by the analysis, we improve this upper bound by introducing a mixed mutation strategy that incorporates both 2-opt moves and permutation jumps. This strategy improves the upper bound to . In the second part of the paper, we use the information gained in the analysis to incorporate domain knowledge to design two fixed-parameter tractable (FPT) evolutionary algorithms for the planar Euclidean TSP. We first develop a EA based on an analysis by M. Theile, 2009, ”Exact solutions to the traveling salesperson problem by a population-based evolutionary algorithm,” *Lecture notes in computer science*, Vol. 5482 (pp. 145–155), that solves the TSP with *k* inner points in generations with probability . We then design a EA that incorporates a dynamic programming step into the fitness evaluation. We prove that a variant of this evolutionary algorithm using 2-opt mutation solves the problem after steps in expectation with a cost of for each fitness evaluation.

## 1 Introduction

In many real applications, the inputs of an NP-hard combinatorial optimization problem may be structured or restricted in such a way that it becomes tractable to solve in practice despite having a worst-case exponential time bound. Parameterized analysis seeks to address this by expressing algorithmic runtime in terms of an additional hardness parameter that isolates the source of exponential complexity in the problem structure. In this paper, we study the application of evolutionary algorithms (EAs) to the Euclidean traveling salesperson problem (TSP) and consider the runtime of such algorithms as a function of both problem size and a further parameter that influences how hard the problem is to solve by an EA.

### 1.1 The Euclidean Traveling Salesperson Problem

Iterative heuristic methods (such as local search and evolutionary algorithms) that rely on the exchange of a few edges such as the well-known 2-opt (or 2-change) operator are popular choices for solving large-scale TSP instances in practice. This is partly due to the fact that they have a simple implementation and typically perform well empirically. However, for these algorithms, theoretical understanding still remains limited. Worst-case analyses demonstrate the existence of instances on which the procedure can be inefficient. Chandra et al. (1999), building on unpublished results due to Lueker, have shown that local search algorithms employing a *k*-change neighborhood operator can take exponential time to find a locally optimal solution. Even in the Euclidean case, Englert et al. (2007) have recently shown that a local search algorithm employing inversions can take worst-case exponential time to find tours that are locally optimal.

If the search operator is restricted to specialized 2-opt moves that remove only edges that intersect in the plane, van Leeuwen and Schoone (1981) proved that a tour that has no such planar intersections can be reached in moves, even if a move introduces further intersecting edges. If the vertices are distributed uniformly at random in the unit square, Chandra et al. (1999) showed that the expected time to find a locally optimal solution is bounded by . More generally, for so-called -perturbed Euclidean instances, Englert et al. (2007) proved that the expected time to find a locally optimum solution is bounded by . These results also imply similar bounds for simple ant colony optimization algorithms as shown in Kötzing et al. (2012).

To allow for a deeper insight into the relationship between problem instance structure and algorithmic runtime, we appeal in this paper to the theory of parameterized complexity (Downey and Fellows, 1999; Flum and Grohe, 2006). Rather than expressing the runtime solely as a function of problem size, parameterized analysis decomposes the runtime into further parameters that are related to the structure of instances. The idea is to find parameters that partition off the combinatorial explosion that leads to exponential runtimes (Downey and Fellows, 1999).

In the context of TSP, a number of parameterized results currently exist. Deneko et al. (2006) showed that, if a Euclidean TSP instance with *n* vertices has *k* vertices interior to the convex hull, there is a dynamic programming algorithm that can solve the instance in time bounded by , where *g* is a function that depends only on *k*. This means that this parameterization belongs to the complexity class FPT, the class of parameterized problems that are considered fixed-parameter tractable. Of course, membership in FPT depends strongly on the parameterization itself. For example, the problem of searching the *k*-change neighborhood for metric TSP is hard for due to Marx (2008). , but the containment is conjectured to be proper (Flum and Grohe, 2006).

### 1.2 Computational Complexity of Evolutionary Algorithms

Initial studies on the computational complexity of EAs consider their runtime on classes of artificial pseudo-Boolean functions (Droste et al., 2002; He and Yao, 2001, 2003; Yu and Zhou, 2006). The goal of these studies is to consider the impact of the different modules of an EA and to develop new methods for their analysis. This early work was instrumental in establishing a rigorous understanding of the behavior of EAs on simple functions, for identifying some classes of problems that simple EAs can provably solve in expected polynomial time (Droste et al., 2002), and for disproving widely accepted conjectures (e.g., that evolutionary algorithms are always efficient on unimodal functions; Droste et al., 1998).

More recently, classical polynomial-time problems from combinatorial optimization such as minimum spanning trees (Neumann and Wegener, 2006, 2007) and shortest paths (Baswana et al., 2009; Doerr et al., 2011; Scharnow et al., 2004) have been considered. In this case, one does not hope to beat the best problem-specific algorithms for classical polynomial solvable problems. Instead, these studies provide interesting insights into the search behavior of these algorithms and show that many classical problems are solved by general-purpose algorithms such as EAs in expected polynomial time.

Research on NP-hard combinatorial optimization problems such as makespan scheduling, covering problems, and multi-objective minimum spanning trees (Neumann, 2007; Witt, 2005) show that EAs can achieve good approximations for these problems in expected polynomial time. In the case of the TSP, Theile (2009) has proved that a EA based on dynamic programming can exactly solve the TSP in at most steps when is allowed to be exponential in *n*. For a comprehensive presentation of the different results that have been achieved see, for example, the recent text of Neumann and Witt (2010).

Algorithmic runtime on NP-hard problems can be studied in much sharper detail from the perspective of parameterized analysis, and this has only recently been started in theoretical work on EAs. Parameterized results have been obtained for the vertex cover problem (Kratsch and Neumann, 2009), the problem of computing a spanning tree with a maximal number of leaves (Kratsch et al., 2010), variants of maximum 2-satisfiability (Sutton et al., 2012), and makespan scheduling (Sutton and Neumann, 2012).

### 1.3 Our Results

In this paper, we carry out a parameterized complexity analysis for evolutionary algorithms on the planar Euclidean TSP. Specifically, using the parameterization of Deneko et al. (2006), we prove upper bounds on the expected time for certain EAs to solve planar TSP instances as a function of *n*, the number of points in the instance, and of *k*, the number of *inner points*, that is, the points that lie on the interior of the convex hull of the planar TSP instance. Our motivation is to use this parameterization from the classical algorithms community to understand the influence of the number of inner points on the runtime of a black box EA, and then to design EAs that exploit this problem structure to obtain fixed-parameter tractable runtimes.

In the first part of the paper, we investigate EAs applied to the planar Euclidean TSP entirely from a black box perspective. We consider the parameterized runtime bounds on algorithms that do not have access to any concrete problem information. Specifically, in the black box scenario, algorithms can only access the fitness value of each candidate solution as measured by tour length. Our results are for the EA which operates on a population of permutations (candidate Hamiltonian cycles) and produces offspring in each generation using a mutation operator based on 2-opt. This analysis provides further insight into the optimization process that allows us to design a mixed mutation operator that uses both 2-opt moves and permutation jumps and improves the upper bound on the expected runtime of the EA. This part of the paper is an extension of work presented at AAAI 2012 (Sutton and Neumann, 2012).

In the second part of the paper, we leave the black box setting and consider EAs that have access to both fitness values and the ordering of the points that lie on the convex hull. This ordering can be precomputed in polynomial time. Using this approach, we design two fixed-parameter tractable (FPT) evolutionary algorithms for the parameterization of Deneko et al. This first algorithm builds on the previous study of Theile (2009) and the general framework for evolutionary algorithms for dynamic programming presented in Doerr et al. (2011). It works with a large population where each individual can be seen as an entry of the dynamic programming table of the classical approach. The population size grows exponentially with the number of inner points and we assume that this result is mainly of theoretical interest.

While the first FPT algorithm is interesting mostly from a theoretical perspective, our second FPT algorithm is highly practical as a heuristic search algorithm. Instead of searching for a permutation of the *n* given points, we apply evolutionary search only to find a good permutation of the *k* inner points. The outer points (lying on the convex hull) have to appear in cycling order according to the convex hull in any optimal solution. For a fixed permutation *x* on the inner points, the best possible tour according to and *x* can be computed in time using dynamic programming. We make use of these ideas and propose to use EAs that work with permutations of the inner points. Our theoretical investigations show that commonly used mutation operators for permutations such as inversions, jumps, and exchanges yield fixed-parameter EAs. Furthermore, our experimental investigations confirm that this approach leads to significantly better results if the number of inner points is not too large. This part of the paper is an extension of work presented at CEC 2013 (Nallaperuma et al., 2013).

This paper is organized as follows. In Section 2 we introduce the problem, state some preparatory material, and discuss parameterized analysis. In Section 3 we study the runtime of a EA on Euclidean TSP instances in a black box setting. We first consider the time it takes for such algorithms to solve instances whose points lie in convex position, that is, have no inner points. We then prove rigorous runtime bounds for the algorithms as a function of the number of inner points in an instance. In Section 4, we design two EAs with access to some instance-specific information and prove that they are fixed-parameter tractable EAs, that is, they solve the planar TSP with *k* inner points in expected time where *g* depends only on *k*. We conclude the paper in Section 5.

## 2 Preliminaries

*V*be a set of

*n*points in the plane labeled as such that no three points are collinear. We consider the complete, weighted Euclidean graph where

*E*is the set of all 2-sets from

*V*. The weight of an edge is equal to

*d*(

*u*,

*v*): the Euclidean distance separating the points. The goal is to find a set of

*n*edges of minimum weight that form a Hamiltonian cycle in

*G*. A candidate solution of the TSP is a permutation . We will sometimes associate a permutation

*x*with its

*linear form*, which is simply the length-

*n*sequence . The Hamiltonian cycle in

*G*induced by a permutation

*x*is the set of

*n*edges The optimization problem is to find a permutation

*x*which minimizes the fitness function

Geometrically, it will often be convenient to consider an edge as the unique planar line segment with end points *u* and *v*. We say a pair of edges and *intersect* if they cross at a point in the Euclidean plane. An important observation, which we state here without proof, is that any pair of intersecting edges form the diagonals of a convex quadrilateral in the plane (see Figure 1).

If and intersect at a point *p*, they form the diagonals of a convex quadrilateral described by points *u*, *s*, *v*, and *t*. Hence edges , , , and form a set of edges that mutually do not intersect.

A tour is called *intersection-free* if it contains no pairs of edges that intersect.

The convex hull of *V* is the smallest convex set containing *V*. A point is called an *inner point* if *v* lies in the interior of the convex hull of *V*. We denote as the set of inner points of *V*. A point is called an *outer point*. The set of such points we denote by . Hence and partition *V*.

Note that every tour for all permutations *x* on *V* corresponds to a set of edges that describe a closed polygon in the plane. The *covertex class* of a polygon is the class of all polygons that have the same vertex set as the given polygon. The following two theorems are due to Quintas and Supnick (1965).

A planar polygon that is shortest of its covertex class and whose vertices are noncollinear cannot intersect itself.

A planar polygon that is shortest of its covertex class and whose vertices are noncollinear must contain the vertices on the boundary of its convex hull in their cyclic order.

We call the unique cyclic order that arises from the points in hull-order. We say a permutation *x* respects hull-order if any two points in the subsequence of *x* induced by are consecutive in *x* only if they are adjacent along the boundary of the convex hull of *V*. Thus, Theorem 2 states that, if *x* is a minimum of *f*, then *x* respects hull-order. We will actually need a slightly stronger result than the one provided by the claim of Theorem 2. This is captured by the following.

If is intersection-free, then *x* respects hull-order.

Lemma 1 follows from arguments in the proof of Theorem 2 above. In particular, Quintas and Supnick (1965) show that if a polygon does not contain the vertices on the boundary of the convex hull in their cyclic order, then the polygon must intersect itself (and hence, by Theorem 1, cannot be optimal). The claim of the lemma follows then by contraposition.

### 2.1 Parameterized Analysis

Parameterized complexity theory is an extension to traditional computational complexity theory in which the analysis of hard algorithmic problems is decomposed into parameters of the problem input. This approach illuminates the relationship between hardness and different aspects of problem structure because it often isolates the source of exponential complexity in NP-hard problems.

A parameterization of a problem is a mapping of problem instances into the set of natural numbers. We are interested in expressing algorithmic complexity in terms of both problem size and the extra parameter. Formally, let *L* be a language over a finite alphabet . A *parameterization* of *L* is a mapping . The corresponding *parameterized problem* is the pair .

For a string , let and . An algorithm deciding in time bounded by is called a *fixed-parameter tractable* (or fpt-) algorithm for the parameterization . Here is an arbitrary but computable function. Similarly, an algorithm that decides a parameterized problem in time bounded by is called an -algorithm.

When working with the runtime of randomized algorithms such as evolutionary algorithms, one is often interested in the random variable *T* which somehow measures the number of generations the algorithm must take to decide a parameterized problem. A randomized algorithm with *expected* optimization time (respectively, ) is a randomized fpt-algorithm (respectively, -algorithm) for the corresponding parameterization .

In the case of the Euclidean TSP on a set of points *V*, we want to express the runtime complexity as a function of *n* and *k* where and *k* is the number of *inner points*: the vertices that lie in the interior of the convex hull of *V*. For the corresponding optimization problem, we are interested in the runtime until the optimal solution is located.

## 3 Black Box EAs

We begin by investigating the parameterized runtime behavior of simple randomized search heuristics on the Euclidean TSP purely from a black box perspective. From this perspective, we assume that the heuristics do not have any access to problem information other than the fitness of an individual as measured by the tour length induced by the corresponding permutation.

### 3.1 Permutation Operators

Randomized search heuristics that are applied to solving TSP instances are usually designed to iteratively search the space of permutations in order to minimize the fitness function defined in Equation (1). Each permutation corresponds to a particular Hamiltonian cycle in the graph. To move through the space of candidate solutions, move or mutation operators are often constructed based on some kind of elementary operations on the set of permutations on *V*. For the black box algorithms, we will consider two such operations: *inversions* and *jumps*, which we define as follows.

In the space of Hamiltonian cycles, the permutation inversion operation is essentially identical to the well-known 2-opt (or 2-change) operation for TSP. The usual effect of the inversion operation is to delete the two edges and from and reconnect the tour using edges and (see Figure 2). Here and subsequently, we consider arithmetic on the indices to be modulo *n*, that is, and . Since the underlying graph *G* is undirected, when , the operation has no effect because the current tour is only reversed. There is also no effect when . In this case, it is straightforward to check that the edges removed from are equal to the edges replaced to create .

*jump*operation transforms permutations into one another by position shifts in their linear form. A permutation

*x*is transformed into a permutation by moving the element in position

*i*in the linear form of

*x*into position

*j*in the linear form of while the other elements between position

*i*and position

*j*are shifted in the appropriate direction. Without loss of generality, suppose . Then,

Since and have the same effect, there are unique jump operations. The jump operator was used by Scharnow et al. (2004) in the context of runtime analysis of EAs on permutation sorting problems.

### 3.2 The EA

*s*is drawn from a Poisson distribution with unit expectation.

*randomized local search*(RLS), which is simply a randomized hill climber in the space of permutations using 2-opt moves. RLS, illustrated in Algorithm 4, operates by iteratively applying random inversion operations to a permutation in order to try to improve the fitness of the corresponding tours. Unlike the EA, RLS can only generate immediate inversion neighbors so it can become indefinitely trapped in local optima (on the other hand, the EA can eventually escape the local optima, but possibly only after a very long time).

*randomized algorithms*. To analyze the run time of such an algorithm, we examine the sequence of best-so-far solutions it discovers during execution as an infinite stochastic process where denotes the best permutation (in terms of fitness) in the population at iteration

*t*. The goal of runtime analysis is to study the random variable that equals the first time

*t*when is a candidate solution of interest (e.g., an optimal solution). The

*optimization time*of a randomized algorithm is the random variable In the case of the EA, this corresponds to the number of generations (iterations of the mutation, evaluation, selection process) that occur before an optimal solution has been introduced to the population. This is somewhat distinct from the traditional measure of the number of explicit calls to the fitness function (Auger and Doerr, 2011; Neumann and Witt, 2010). However, in the case of the EA, this measurement can be obtained from

*T*by , since we need fitness function calls to evaluate the initial population and each generation requires evaluating an additional individuals. We discuss this further in Section 3. In this paper, we will estimate the

*expected optimization time*of the EA. This quantity is calculated as , the expectation of

*T*. However, we will also report the bounds on , the expected number of fitness evaluations needed until a solution is found.

*n*and the number of inner points

*k*, we analyze a Markov chain generated by the algorithm. We construct the Markov chain as follows. Given a point set

*V*, each permutation

*x*on

*V*that corresponds to a tour that is non-optimal is a unique state in the Markov chain. Finally, every permutation that corresponds to an optimal tour in

*V*is associated with a single absorbing state . We then bipartition the state space (minus ) into two sets and where and Note that since the optimal tour cannot contain intersections. In terms of the Markov chain, the optimization time

*T*is the first hitting time of the state . We will need the following two preparatory lemmas.

*z*with , it will never return to a state in (since such as state has strictly worse fitness). We define a nonnegative distance function as For all , . Moreover, . Hence .

Let be the stochastic process that is a restriction of constructed by taking only permutations in the same order. We consider the drift of the random sequence .

*c*is bounded below by

*p*. Therefore, we have By the additive drift theorem (He and Yao, 2001), the random variable is bounded above by . Finally, the series on the LHS of the claimed inequality is the total time the algorithm spends in , or more precisely, and hence the claim follows.

Again, since for any there is a nonzero probability the algorithm exits the state *x* and transits to the absorbing state . This event is characterized as an independent Bernoulli trial with success probability at least *q*. It follows that the series in the claim can be estimated by a geometrically distributed random variable with parameter *q*, and therefore the expected time spent in states contained in the partition is bounded above by .

In Section 3.3 we will introduce a constraint on the angles between edges that give rise to suitable values for *c* in the claim of Lemma 2. This, along with corresponding values for *p* and *q* (that appear in the claims of Lemmas 2 and 3, respectively), will allow us to bound the expected runtime in terms of *n* and *k*. Before doing so, it will be convenient to state a number of lemmas that will be useful in the analysis. To preserve the flow of the text, we defer their proofs to the appendix.

Let *x* be a permutation such that is not intersection-free. Then there exists an inversion that removes a pair of intersecting edges and replaces them with a pair of non-intersecting edges.

(see Appendix).

Note that it is still possible that the introduced edges intersect with some of the remaining edges in .

Suppose . Then there are at most distinct intersection-free tours.

(see Appendix).

Suppose and is an intersection-free tour on *V*. Then there is a sequence of at most *k* jump operations that transforms *x* into an optimal permutation. Similarly, there is a sequence of at most inversion operations that transform *x* into an optimal permutation.

(see Appendix).

### 3.3 Angle-Bounding, Quantization, and Resulting Structural Properties

A challenge to the runtime analysis of algorithms that employ edge exchange operations such as 2-opt is that, when points are allowed in arbitrary positions, the minimum change in fitness between neighboring solutions can be made arbitrarily small. Indeed, proof techniques for worst-case analysis often leverage this fact (Englert et al., 2007). To circumvent this, we impose bounds on the angles between points, which allows us to express runtime results as a function of trigonometric expressions involving these bounds. Momentarily, we will refine this further by introducing a class of TSP instances embedded in an grid. In that case, we will see that the resulting trigonometric expression is bounded by a polynomial in *m*.

We say *V* is *angle-bounded by * for some if for any three points , where denotes the angle formed by the line from *u* to *v* and the line from *v* to *w*. This allows us to express a bound in terms of on the change in fitness from a move that removes an inversion.

*V*is angle-bounded by . Let

*x*be a permutation such that is not intersection-free. Let be the permutation constructed from an inversion on

*x*that replaces two intersecting edges in with two non-intersecting edges.

^{1}Then, if

*d*denotes the minimum distance between any two points in

_{min}*V*, .

(see Appendix).

The lower bound on the angle between any three points in *V* provides a constraint on how small the change in fitness between neighboring inversions can be. This lower bound is useful in the case of a *quantized* point set. This occurs when the points can be embedded on an grid as illustrated in Figure 5.

Quantization, for example, occurs when the *x* and *y* coordinates of each point in the set are rounded to the nearest value in a set of *m* equidistant values (e.g., integers). We point out that it is still important that the quantization preserve the constraint on collinearity since collinear points violate a nonzero angle bound. We have the following lemma.

Suppose *V* is a set of points that lie on an unit grid, no three collinear. Then *V* is angle-bounded by .

(see Appendix).

Lemma 8 allows us to translate the somewhat awkward trigonometric expression in the claim of Lemma 7 (and subsequent lemmas that depend on it) into a convenient polynomial that can be expressed in terms of *m*.

Let *V* be a set of *n* points that lie on an unit grid, no three collinear. Then, *V* is angle-bounded by where .

(see Appendix).

We are now ready to prove the following technical lemma for 2-opt mutation (see Algorithm 3). This lemma will be instrumental in proving runtime bounds later in the paper.

Let *V* be a set of planar points in convex position angle-bounded by . We have the following.

1. For any , the probability that 2-opt mutation creates an offspring *y* with is at least .

2. For any , the probability that 2-opt mutation creates an optimal solution is at least

For (1), suppose . By Lemma 4, there is at least one pair of intersecting edges in that can be removed with a single inversion operation . Let . By Lemma 7, satisfies the fitness bound in the claim. It suffices to bound the probability that *y* is produced by 2-opt-mutation (*x*).

Let *E*_{1} denote the event that Poisson mutation performs exactly one inversion (i.e., *s* = 0 in line 2 of Algorithm 3). Let *E*_{2} denote the event that the pair is specifically chosen for the inversion.

*y*from

*x*is

For (2), suppose . Thus, is intersection-free, and it follows from Lemma 6 that there are at most inversion moves that transform *x* into an optimal solution.

Let denote the event that Poisson mutation performs exactly inversions (i.e., in line 2 of Algorithm 3). Let denote the event that all inversions are the correct moves that transform *x* into an optimal solution.

Finally, since the time bounds in the remainder of this paper are expressed as a function of the angle bound , and the bounds on point distance, it will be useful to define the following function.

### 3.4 Instances in Convex Position

A finite point set *V* is in *convex position* when every point in *V* is a vertex on its convex hull. Deneko et al. (2006) observed that the Euclidean TSP is easy to solve when *V* is in convex position. In this case, the optimal permutation is any linear ordering of the points which respects the ordering of the points on the convex hull. Such an ordering can be found in time (de Berg et al., 2008).

In the context of EAs the natural question arises, if *V* is in convex position, how easy is it for simple randomized search heuristics? In this case, a tour is intersection-free if and only if it is globally optimal, hence finding an optimal solution is exactly as hard as finding an intersection-free tour. Since an intersection can be (at least temporarily) removed from a tour by an inversion operation (cf. Lemma 4), we focus in this section on algorithms that use the inversion operation.

We begin our analysis of instances in convex position by considering RLS. RLS operates on a single candidate solution by performing a single inversion in each iteration. Since each inversion which removes an intersection results in a permutation whose fitness is improved by the amount bounded in Lemma 7, it is now straightforward to bound the time it takes for RLS to discover a permutation that corresponds to an intersection-free tour.

Let *V* be a set of planar points in convex position angle-bounded by . The expected time for RLS to solve the TSP on *V* is where *A* is defined in Definition 6.

*T*defined in Equation (2). Since

*V*is in convex position, any intersection-free tour is globally optimal. Thus, in this case , and Equation (3) can be written as Consider an arbitrary permutation . Since must contain intersections, by Lemma 4, there is an inversion which removes a pair of intersecting edges and replaces them with a pair of non-intersecting edges. Moreover, by Lemma 7, such an inversion results in an improvement of at least If the state

*x*is visited by RLS, the probability that this particular inversion is selected uniformly at random is . Thus, we have the conditions of Lemma 2 with and

*c*equal to the expression in equation (4) and the claimed bound on the expectation of

*T*follows.

If *V* is in convex position and embedded in an grid with no three points collinear, then RLS solves the TSP on *V* in expected time .

The bound follows immediately from Theorem 3 since *V* is in convex position and since, by Lemma 8, *V* is angle-bounded by , , and . Appealing to Lemma 9 yields . Substituting these terms into the bound of Theorem 3 completes the proof.

We now consider the optimization time of the EA using 2-opt mutation defined in Algorithm 1 applied to a set of points in convex position. Obviously, in this case we will find and terms appearing in the runtime formulas. We will assume that and are polynomials in both *n* and *k*. We will also find that setting ensures that a transition from any state to a state which improves on the fitness by at least a specified amount occurs with constant probability.

Such a setting has the effect of reducing the number of *generations* spent removing intersections from the best-so-far tours. However, it is important to note that the number of calls to the fitness function must be accordingly increased by a factor of in each generation. Nevertheless, the expected number of generations can be a useful measure when considering parallel EAs. As Jansen et al. (2005) have pointed out, when the fitness evaluation of the offspring can be performed on parallel processors, the number of generations corresponds to the *parallel* optimization time. In such a case, we would observe a quadratic factor improvement in the parallel runtime corresponding to the segment spent in tours with intersections. Nevertheless, we also report the bound on the number of total fitness function evaluations.

The following theorem bounds the number of expected generations that the EA needs to solve the Euclidean TSP on a set of angle-bounded points in convex position.

Let *V* be a set of planar points in convex position angle-bounded by . The expected time for the EA using 2-opt mutation to solve the TSP on *V* is bounded above by where *A* is as defined in Definition 6.

*T*defined in Equation (2). Again, in the case of convex position, there are no non-optimal intersection-free tours so that and As long as is non-optimal, must contain at least one pair of intersecting edges. Hence, in generation

*t*, if is selected for mutation to create one of the offspring, then by Lemma 10, 2-opt mutation must improve the best-so-far solution by at least with probability at least . The probability that at least one of the offspring improves on by at least this amount is We now make the following case distinction on .

*Case 1:*. For this setting of , we have , so an intersection is removed in each generation with constant probability. Invoking Lemma 2, the expected time to find an intersection-free tour is at most .

*Case 2:*. Here we have The final inequality comes from the fact that for . Thus, in this case we have . Applying Lemma 2, the expected time to find an intersection-free tour is at most .

Thus, the expected time to solve the instance is bounded by which yields the claimed bound.

We immediately get the following corollary.

Let *V* be a set of planar points in convex position angle-bounded by . The expected number of fitness evaluations needed by the EA using 2-opt mutation to solve the TSP on *V* is bounded above by , and the expected optimization time of the EA is bounded above by , where *A* is as defined in Definition 6.

Moreover, analogous to the corollary to Theorem 3, we have the following.

If *V* is in convex position and embedded in an grid with no three points collinear, then the EA solves the TSP on *V* in expected time

### 3.5 Instances with *k* Inner Points

We now turn our attention to TSP instances in which and express the expected runtime in terms of , the number of points, and , the number of inner points.

#### 3.5.1 2-opt Mutation

We use the structural analysis in Section 3.3 to show that when there are few inner points, intersection-free tours are somehow “close” to an optimal solution in the sense that relatively small perturbations by the EA suffice to solve the problem. Again, recall that we assume and are polynomials in both *n* and *k*.

Let *V* be a set of points angle-bounded by such that . The expected time for the EA using 2-opt mutation to solve the TSP on *V* is bounded above by .

We argue by analyzing the Markov chain generated by the EA using 2-opt mutation. Let denote the sequence of best-so-far states visited by the EA.

*t*, if , then the probability of generating an offspring that improves the fitness by at least is identical to that in the proof of Theorem 4. Arguing in the same manner as in the proof of Theorem 4, we have

*q*into the claim of Lemma 3, we have The bound on then follows from Equation (3) and linearity of expectation.

We also get the following corollary.

Let *V* be a set of points angle-bounded by such that . The expected number of fitness evaluations needed for the EA using 2-opt mutation to solve the TSP on *V* is bounded above by and the expected optimization time for the (1 + 1) EA is .

Again, from Lemmas 8 and 9 we also have the following corollary.

Let *V* be a set of points quantized on an grid such that . The expected time for the EA using 2-opt mutation to solve the TSP on *V* is .

#### 3.5.2 Mixed Mutation Strategies

Similar to Lemma 10, we have the following result for mixed mutation.

Let *V* be a set of planar points in convex position angle-bounded by . Then,

1. For any , the probability that mixed mutation creates an offspring *y* with is at least .

2. For any , the probability that mixed mutation creates an optimal solution is at least .

For (1), the probability that mixed mutation selects inversions is and the rest of the claim follows from the argument of Lemma 10.

For (2), suppose . Thus, is intersection-free, and it follows from Lemma 6 that there are at most *k* jump operations that transform *x* into an optimal solution. The remainder of the proof is identical to that of Lemma 10. The probability that mixed mutation selects jump operations contributes the leading factor of .

Let *V* be a set of points angle-bounded by such that . The expected time for the EA using mixed mutation to solve the TSP on *V* is bounded above by .

The proof is identical to the proof of Theorem 5, except that we substitute the probabilities from Lemma 11 into Lemmas 2 and 3.

We immediately get the following corollary.

Let *V* be a set of points angle-bounded by such that . The expected number of fitness evaluations needed for the EA using mixed mutation to solve the TSP on *V* is bounded above by and the expected optimization time for the (1 + 1) EA is bounded above by .

As before, from Lemmas 8 and 9 we have the following.

Let *V* be a set of points quantized on an grid such that . The expected time for the EA using mixed mutation to solve the TSP on *V* is .

## 4 Fixed-Parameter Evolutionary Algorithms

Moving away from the black box perspective, we now consider designing EAs that are given access to a small amount of problem information. Specifically, we allow the algorithms access to the ordering of points that lie on the convex hull. Using this information, we design two fixed-parameter tractable EAs for the inner point parameterization of Deneko et al. (2006).

### 4.1 A Population-Based Approach

In this section we describe a EA that solves the TSP to optimality in fixed-parameter tractable time. We build on the evolutionary dynamic programming framework for TSP introduced by Theile (2009). We show that a slight modification to Theile’s EA that carefully maintains the invariant that the points in remain in convex-hull order for each individual allows us to improve the time bound significantly on instances for which the number of inner points is not too large.

*V*, with and , let denote the order on given by Definition 7. A permutation

*x*on a subset

*U*of

*V*is a bijection . The EA is described in Algorithm 6. The EA maintains a population

*P*of permutations on subsets of

*V*that have a special structure that we now describe. Let , let , and let . An individual is a permutation on with the following properties.

1. .

2. .

3. *x* is -respecting, that is, appear in order.

For a permutation , we call the set its *ground set* and we call *r* its *tail vertex*.

The TSP tour defined by a permutation is a cycle , that is, it starts at *p*_{1}, runs over all nodes in such that any nodes in are visited in the order they appear in , and finally visits *r* before returning to *p*_{1}. The full population of the EA consists of an individual for every , every , and possible tail vertex *r*.

*P*and an offspring is created for each individual by applying a mutation operator. This mutation operator attempts to extend each individual by “growing” its ground set. To mutate a single individual , a vertex

*v*is chosen uniformly at random from the remaining vertices in . A new permutation is created by concatenating

*v*to the linear order described by

*x*, that is, for , Thus is defined on a different (slightly larger) ground set than

*x*using as the new tail vertex. Following our notation, we have Obviously, when and , the mutation operator has no effect, since the ground set cannot be extended for such an individual.

In order to introduce sufficiently large changes to the population in each generation, we utilize the Poisson mutation strategy in the same manner as both Theile’s original algorithm and the black box EAs introduced in Section 3. Specifically, in each generation, for each of the selected individuals, we apply the mutation operator defined above times, where *s* is drawn from a Poisson distribution with unit expectation.

Finally, we again employ truncation selection in the sense that each mutated offspring may only replace the individual in the parent population with the same ground set and tail vertex. This replacement occurs only if the fitness of the mutated offspring is at least as good as the parent individual in question. Thus, at the end of a generation, the parent population size is preserved.

We now bound the expected time until the EA described in Algorithm 6 has solved a TSP instance on *n* points with *k* inner points to optimality. We first bound the population size with the following lemma.

The parent population size is bounded above by .

The population is initialized by constructing, for each possible distinct ground set and each legal tail vertex *r* a length permutation such that , , and the elements from appear in the same order as they do in . Note that to maintain the correct ordering, the tail vertex cannot be any point *p _{j}* for , or the ordering with respect to is violated. Therefore, the tail vertex can be any member of .

After the population *P* is initialized, for each ground set and each legal tail vertex , there exists some permutation that respects the order .

*p*

_{1}and ending at

*r*, visiting each point in exactly once, and respecting the order . Because each Hamiltonian path possesses the optimal substructure property (i.e., subpaths of optimal paths are also optimal paths),

*F*can be defined recursively using the Bellman principle (Bellman, 1962; Deneko et al., 2006), Thus, if there is a permutation with an optimal subpath in the population, there exists some correct mutation that can construct a longer path that is also optimal. Furthermore, a permutation that represents a suboptimal path through the ground set can always be replaced by one that represents an optimal subpath through that ground set, that is, some such that by the selection operation. Hence, once an optimal subpath is found for a particular ground set and tail vertex, the subpath optimality is never lost. To bound the optimization time of the EA, it suffices to bound the time until a particular length-

*n*permutation has its optimal path in the population.

The proof of the runtime is a straightforward extension of Theile's (2009) argument that the EA can simulate dynamic programming on a function similar to *F* in a randomized fashion.

Let *V* be a set of *n* points in the Euclidean plane with . After generations, the EA has solved the TSP on *V* to optimality in expectation and with probability .

Suppose there exists an individual such that . Since selection for reproduction occurs uniformly at random, with probability at least , *x* is selected for reproduction to produce one of the offspring by mutation in each subsequent generation. Furthermore, with probability , only a single mutation step is performed on *x* (i.e., following line 11 of Algorithm 6). Finally, the probability that *x* is extended in an optimal fashion is at least , that is, the probability that the correct vertex *q* is chosen out of to create an optimal subpath on the ground set .

*m*by applying the correct mutation to the parent individual representing optimal subpath of length . We have In the base case, the permutation corresponds to an optimal length-1 path and is present after initialization in lines 1–2 of Algorithm 6. Since optimal paths through a given ground set and tail vertex are never lost, it follows by induction that the expected number of generations until an optimal path of length

*m*exists in the population is at most . Thus, the expected number of generations until an optimal solution enters the population is bounded above by . We make a case distinction on .

*Case 1:*. In this case, we have again since for . The expected number of generations until the optimal solution is found is at most In the last step, we have applied the result of Lemma 12.

*Case 2:*. For this setting of we have and the number of generations until the optimal solution is found is at most .

To prove the concentration result, for some constant and any , let be the number of generations needed to produce an optimal subpath of length *m* where . We define a sequence of random indicator variables for all such that if the correct mutation has occurred to extend an optimal subpath in the *i*th generation. Let . The expectation of *X* is , by the above bounds on .

*t*fitness evaluations, and the probability of this event is at most by Chernoff bounds. Suppose there are individuals in the population that are length

*m*. Clearly . The probability that at least one of the individuals in

*P*has not been optimized within

*t*generations is, by the union bound, at most where is a constant. This completes the proof.

The expected number of fitness evaluations is .

### 4.2 Searching for Permutations of Inner Points

In this section, we propose an approach that makes use of insights in the context of parameterized complexity. By Theorem 2, the outer points are always visited in hull-order in every optimal solution. In the following, we assume that is the fixed order on the outer points given in Definition 7. Let *x* be a permutation on the inner points . Using the dynamic programming algorithm of Deneko et al., one can compute the shortest Hamiltonian cycle respecting and *x* in time .

These ideas suggest that using a heuristic to only search for an optimal permutation on the inner points can significantly reduce the effective size of the search space, and thus lead to quicker solution times for EAs. For each permutation on the inner points *x*, the quality can be evaluated by running a dynamic programming algorithm to compute the best possible tour using and *x*.

Let be the fixed order of the outer points and be an arbitrary permutation of the inner points. We can compute the shortest tour respecting and *x* using the dynamic programming approach of Deneko et al. (2006), which we state in the following.

The dynamic programming algorithm computes a three-dimensional table *F* with entries where , , and . The algorithm computes for each triple the cost of the shortest path from *p*_{1} through all nodes and respecting the orders and *x*. The path ends at *p _{i}* iff determines that the last node is an outer point and ends at

*q*iff determines that the last node is an inner point.

_{j}The time computing the *F* array is proportional to its size which is . Therefore, the fitness for a given permutation *x* of the inner points can be computed in time .

*s*is chosen according to the Poisson distribution with unit expectation. Then truncation selection is applied to preserve the best individuals out of all individuals for the next generation.

For mutation, we employ the inversion (2-opt) and permutation jump mutation operators introduced in Section 3.1. In the case of inversions, however, it is no longer clear whether this is the correct choice for the new search space, which is restricted only to orderings on the inner points. In addition to inversions and jumps (see Definitions 2 and 3), we also consider the *exchange* operator, which exchanges two elements (chosen uniformly at random) in the ordering of a permutation. We formalize the exchange operator in the following definition.

We denote by () EA, () EA, and () EA, the variants of the () EA using inversions, jumps, and exchanges, respectively.

In the following, we investigate the number of operations needed in order to obtain an optimal solution in dependence of the chosen mutation operator. This will lead to different upper bounds on the runtime of the algorithm. For all the operators, we show that they yield fixed-parameter tractable results.

Let *V* be a set of *n* points in the Euclidean plane with . The expected number of generations needed to solve the TSP on *V* for both the () EA, and the () EA is bounded by . For the () EA, the bound is .

Let be a lower bound on the probability to transform an arbitrary non-optimal solution into an optimal solution. Suppose there are no optimal solutions in the population. The probability that at least one of the offspring is optimal is bounded below by .

*k*elements into any other permutation on the same elements is at most (this is due to a conjecture by Gollan that was later proved by Bafna and Pevzner, 1996. For the jump and exchange operators, the number of these operations required is obviously at most

*k*. The claimed bounds on the expected number of generations for each variant follow from these arguments, and the proof is complete.

As usual, the expected number of fitness evaluations can be derived by where is bounded by Theorem 8.

The previous theorem shows that simple EAs are fixed-parameter tractable EAs for the considered problem when working with permutations on the inner points and applying dynamic programming as part of the fitness evaluation. Note that dynamic programming takes time . This is more expensive than computing the cost of a given tour on *n* cities, which can be done in time. However, it is still feasible to use the dynamic programming approach as part of each fitness evaluation. In the next section, we show that this also leads to a significantly better optimization process than working with the standard approach that searches through the set of permutations on all *n* cities.

### 4.3 Experiments

*n*points. We denote by (1 + 1) EA, (1 + 1) EA, and (1 + 1) EA, the variants of (1 + 1) EA using inversions, jumps, and exchanges, respectively.

We consider several planar Euclidean TSP instances of various sizes (25, 50, and 100) and, within each size class, control the percentages of inner points. The instance generation process is described in detail in the next section. We calculate the solution quality reached within a given time limit on each instance. On these instances, we run the EAs using each mutation strategy for both the (1 + 1) EA and the (1 + 1) EA. As a baseline, we also compute the value of the optimal solution on each instance using the exact TSP solver Concorde (Applegate et al., n.d.).

For each run, we set time limits based on the instance size. Hence, we run each EA for 1 s on instances with 25 points, 3 s for instances with 50 points, and finally 15 s for instances with 100 points. We measure the *iteration count* which is the number of fitness function calls for each EA. The (1 + 1) EA runs roughly 200,000 iterations within the given time limits for all instance sizes. On the other hand, the (1 + 1) EA performs in the range of 1000–5000 iterations. The number of iterations performed by the (1 + 1) EA decreases as a function of the ratio of outer points to inner points in the instance. The experiments are conducted on a PC with 2.4 GHz Intel Core i5 CPU and 4GB 1,333 MHz DDR3 RAM.

#### 4.3.1 Parameterized Instances

Since we need to study the impact of the number of inner points on algorithm performance, we cannot use standard benchmark instances, since it is not clear how to cleanly control the number of inner points in such problem sets. Instead, we generate random instances controlling for the percentage of points that lie in the interior of the convex hull. For each size category , we generate a set of 30 instances each with inner point percentages 5%, 10%, 20%, 30%, and 40%. For example, the set contains 30 TSP instances on planar points such that exactly lie in the interior of the convex hull. To construct such instances, we first generate the outer points on a convex hull, and then add the required number of inner points. We create a circle with a given radius and randomly generate outer points on the circumference by producing a random angle and then deriving *x* and *y* coordinates from the angle with respect to the *y* axis. This process continues until we have constructed the required values for . We then identify the convex hull and generate random points inside the hull within a distance of zero to the radius. Here, we accept a randomly generated inner point only if it falls inside the convex hull.

#### 4.3.2 Results

We measured the solution quality obtained within the allocated time for the six (1 + 1) EA variants described in this paper. We also compared each with the optimal solution computed by Concorde. The dataset consists of 30 instances from each group representing each instance size and each inner node percentage.

The results suggest that the (1 + 1) EA approach consistently leads to more effective optimization when there are not too many inner points. In Table 1, we see the average values of solution quality obtained for the instances in each instance size and inner point percentage. For some cases, variants of (1 + 1) EA (e.g., the set of with 5% inner points) reach the optimal values computed by the exact solver Concorde, while in most instances, they optimize the instances more effectively than the corresponding variants that search through permutations on all *n* points ((1 + 1) EA). In a few cases, where the inner point percentage is high, the new approach tends to perform poorly. This is expected, since the construction procedures are leveraging the constraints on the convex hull. Therefore, we expect the performance to degrade as the number of inner points becomes large with respect to the problem size.

Instance . | Exchanges . | Inversions . | Jumps . | . | |||
---|---|---|---|---|---|---|---|

n/% inner
. | (1 + 1) EA . | (1 + 1) EA . | (1 + 1) EA . | (1 + 1) EA . | (1 + 1) EA . | (1 + 1) EA . | Concorde . |

25/5% | 81,937 | 53,141 | 53,665 | 53,141 | 59,173 | 53,141 | 53,141 |

25/10% | 82,592 | 57,904 | 59,190 | 57,896 | 66,070 | 57,483 | 57,482 |

25/20% | 86,646 | 63,005 | 64,222 | 63,132 | 70,232 | 61,787 | 61,324 |

25/30% | 83,869 | 67,864 | 67,570 | 67,325 | 72,586 | 65,761 | 64,165 |

25/40% | 84,941 | 70,912 | 68,486 | 69,363 | 72,649 | 67,229 | 65,370 |

50/5% | 128,640 | 61,090 | 63,258 | 61,098 | 73,293 | 60,936 | 60,936 |

50/10% | 131,391 | 67,155 | 69,625 | 67,012 | 79,230 | 66,389 | 65,831 |

50/20% | 131,027 | 79,682 | 81,414 | 78,429 | 87,586 | 77,019 | 74,809 |

50/30% | 133,533 | 89,645 | 86,474 | 86,662 | 95,599 | 83,730 | 79,785 |

50/40% | 131,902 | 97,231 | 89,271 | 92,597 | 98,999 | 90,102 | 82,469 |

100/5% | 210,234 | 69,996 | 74,392 | 69,895 | 81,839 | 69,466 | 68,990 |

100/10% | 211,502 | 83,840 | 86,301 | 82,720 | 95,024 | 81,613 | 79,529 |

100/20% | 208,611 | 104,295 | 100,239 | 98,373 | 113,363 | 98,855 | 90,840 |

100/30% | 215,322 | 121,505 | 109,335 | 112,168 | 125,907 | 110,406 | 98,853 |

100/40% | 212,083 | 138,134 | 117,594 | 124,852 | 135,720 | 128,331 | 105,432 |

Instance . | Exchanges . | Inversions . | Jumps . | . | |||
---|---|---|---|---|---|---|---|

n/% inner
. | (1 + 1) EA . | (1 + 1) EA . | (1 + 1) EA . | (1 + 1) EA . | (1 + 1) EA . | (1 + 1) EA . | Concorde . |

25/5% | 81,937 | 53,141 | 53,665 | 53,141 | 59,173 | 53,141 | 53,141 |

25/10% | 82,592 | 57,904 | 59,190 | 57,896 | 66,070 | 57,483 | 57,482 |

25/20% | 86,646 | 63,005 | 64,222 | 63,132 | 70,232 | 61,787 | 61,324 |

25/30% | 83,869 | 67,864 | 67,570 | 67,325 | 72,586 | 65,761 | 64,165 |

25/40% | 84,941 | 70,912 | 68,486 | 69,363 | 72,649 | 67,229 | 65,370 |

50/5% | 128,640 | 61,090 | 63,258 | 61,098 | 73,293 | 60,936 | 60,936 |

50/10% | 131,391 | 67,155 | 69,625 | 67,012 | 79,230 | 66,389 | 65,831 |

50/20% | 131,027 | 79,682 | 81,414 | 78,429 | 87,586 | 77,019 | 74,809 |

50/30% | 133,533 | 89,645 | 86,474 | 86,662 | 95,599 | 83,730 | 79,785 |

50/40% | 131,902 | 97,231 | 89,271 | 92,597 | 98,999 | 90,102 | 82,469 |

100/5% | 210,234 | 69,996 | 74,392 | 69,895 | 81,839 | 69,466 | 68,990 |

100/10% | 211,502 | 83,840 | 86,301 | 82,720 | 95,024 | 81,613 | 79,529 |

100/20% | 208,611 | 104,295 | 100,239 | 98,373 | 113,363 | 98,855 | 90,840 |

100/30% | 215,322 | 121,505 | 109,335 | 112,168 | 125,907 | 110,406 | 98,853 |

100/40% | 212,083 | 138,134 | 117,594 | 124,852 | 135,720 | 128,331 | 105,432 |

To further investigate the effect of mutation strategy, we examine the three variants of the (1 + 1) EA for the problem set and 40% inner points. The solution quality reached in the allocated time (15 s) is plotted as a function of instance for each variant in Figure 6.

To determine whether the (1 + 1) EA outperforms the (1 + 1) EA with statistical significance, we perform a one-tailed Wilcoxon signed rank test (Wilcoxon, 1945) on the solution quality measurements obtained after the allocated runtime. The resulting positive rank sum (*W*) and confidence (*p* values) are recorded. We perform the test for each mutation strategy on each instance, and observe the effect of inner point percentage and size. To perform these analyses, we use the GNU R software environment (R Development CoreTeam, 2011). The results are summarized in Table 2 where the control variables (instance size and inner point percentage) are indicated in the first column.

Instance . | Exchanges . | Inversions . | Jumps . | |||
---|---|---|---|---|---|---|

n/% inner
. | W
. | p
. | W
. | p
. | W
. | p
. |

25/5% | 465 | * | 91 | * | 253 | * |

25/10% | 465 | * | 336 | * | 465 | * |

25/20% | 465 | * | 366 | 0.003 | 465 | * |

25/30% | 465 | * | 293 | 0.11 | 465 | * |

25/40% | 465 | * | 172 | 0.894 | 465 | * |

50/5% | 465 | * | 335 | * | 435 | * |

50/10% | 465 | * | 435 | * | 465 | * |

50/20% | 465 | * | 431 | * | 465 | * |

50/30% | 465 | * | 217 | 0.627 | 465 | * |

50/40% | 465 | * | 33 | 1 | 465 | * |

100/5% | 465 | * | 460 | * | 465 | * |

100/10% | 465 | * | 462 | * | 465 | * |

100/20% | 465 | * | 399 | * | 465 | * |

100/30% | 465 | * | 147.5 | 0.960 | 461 | * |

100/40% | 465 | * | 56 | 1 | 394 | * |

Instance . | Exchanges . | Inversions . | Jumps . | |||
---|---|---|---|---|---|---|

n/% inner
. | W
. | p
. | W
. | p
. | W
. | p
. |

25/5% | 465 | * | 91 | * | 253 | * |

25/10% | 465 | * | 336 | * | 465 | * |

25/20% | 465 | * | 366 | 0.003 | 465 | * |

25/30% | 465 | * | 293 | 0.11 | 465 | * |

25/40% | 465 | * | 172 | 0.894 | 465 | * |

50/5% | 465 | * | 335 | * | 435 | * |

50/10% | 465 | * | 435 | * | 465 | * |

50/20% | 465 | * | 431 | * | 465 | * |

50/30% | 465 | * | 217 | 0.627 | 465 | * |

50/40% | 465 | * | 33 | 1 | 465 | * |

100/5% | 465 | * | 460 | * | 465 | * |

100/10% </ |