Metaheuristics are an effective and diverse class of optimization algorithms: a means of obtaining solutions of acceptable quality for otherwise intractable problems. The selection, construction, and configuration of a metaheuristic for a given problem has historically been a manually intensive process based on experience, experimentation, and reasoning by metaphor. More recently, there has been interest in automating the process of algorithm configuration. In this article, we identify shared state as an inhibitor of progress for such automation. To solve this problem, we introduce the Automated Open-Closed Principle (AOCP), which stipulates design requirements for unintrusive reuse of algorithm frameworks and automated assembly of algorithms from an extensible palette of components. We demonstrate how the AOCP enables a greater degree of automation than previously possible via an example implementation.
Metaheuristics define families of algorithms for obtaining acceptable solutions to hard optimization problems. Each metaheuristic defines a general framework for optimization that must be tailored to individual applications in order to exploit domain-specific information or to incorporate novel search mechanisms. A practitioner must select an appropriate metaheuristic, customize it in terms of its constituent components—such as functions implementing perturbation operators or termination criteria—and specify many parameter settings. For example, they may select a suitable heuristic for recombining two solutions to produce another solution, or set the rate of crossover in a Genetic Algorithm (Luke, 2013). The practitioner is thus faced with a hyper-parameter optimization problem in the design space of metaheuristics.
To solve this manual design problem, practitioners have previously relied upon anecdotal evidence, trial-and-error methods such as “one factor at a time” parameter tuning, and “reasoning by metaphor.” This has led to a plethora of algorithms, components, and parameters, divided into separate research silos. As new extensions are made to existing algorithms, the design space continues to expand; concepts from one family of metaheuristics are reinvented elsewhere (Weyland, 2010). This manual process of development is fundamentally ad hoc, does not promote reusability of components or separation of concerns, and does not scale.
The development of flexible frameworks that facilitate the reusability of components, the incremental design of metaheuristics, and ultimately the automation of this manual design process, is thus an important challenge for metaheuristics research. Whilst related work such as “Programming by Optimization” (Hoos, 2012) and configurators (Hutter et al., 2009; Ansótegui et al., 2009; Hutter et al., 2011a; López-Ibáñez et al., 2016) have supported the optimization of algorithm parameters and even the exploration of restricted combinatorial design spaces (KhudaBukhsh et al., 2009; López-Ibáñez and Stützle, 2012; Mascia et al., 2014; Adriaensen et al., 2014; Bezerra et al., 2016), they are limited in their generality by the prevalence of state dependencies between components; new components cannot be added without modifying the grammar that is provided to a configurator and/or changing the call graph between components. It is these state dependencies that prevent a clean separation of concerns in metaheuristic design.
In this article, we introduce the Automated Open-Closed Principle (AOCP), stipulating the properties required for extensible combinatorial assembly. We describe an implementation of an example metaheuristic application, using principled state threading to manage state dependencies. Our main contributions are as follows:
We describe how state dependencies form an obstacle to composition and reuse of components, and to correspondingly greater automation of algorithm design.
We propose the AOCP which stipulates requirements for overcoming this obstacle.
We provide an example implementation using functional programming constructs that demonstrates the efficacy of the AOCP.
We discuss the other advantages of following the AOCP in automated algorithm design.
2 State Dependencies between Metaheuristic Components
Given the role of metaheuristics as generic problem-solving strategies, there is a natural desire to parameterize metaheuristic frameworks by a relatively small number of components such as perturbation operators, selection methods, and termination criteria. As discussed in detail in subsequent sections, a genuinely clean separation of concerns (both between framework and components and between components themselves) is difficult to obtain because of state dependencies. In particular, the lack of a principled means of handling such dependencies has inhibited progress towards automated assembly, particularly at larger scales. This problem is particularly acute when considering the addition of new components to an existing system, whereby the original system must be modified to allow for their inclusion. This modification violates the “Open-closed principle” (OCP) of framework design (Larman, 2001): an implementation should be closed to modification but nonetheless open to extension via custom components. Crucially, it inhibits reusability of components and forestalls the possibility of the automated exploration of the design space.
As a concrete example of this difficulty, Listing 1 gives a simple local search framework that allows for three design decisions, namely the choice of perturbation, acceptance, and termination conditions. Each specific triple of components (perturb, accept, isFinished) used to configure the framework corresponds to a specific local search algorithm. This allows us to concisely specify a large design space as the Cartesian product of alternative design decisions, and also exposes the relationship between different designs in terms of decisions that they have in common. We can further generalize by parameterizing each component in turn by its own design choices. This design space could be automatically explored using a configurator approach; configurators typically require a grammar to be provided in a configuration file, which would necessitate a second specification of this design space.
To permit the substitution of different choices for each component, they must conform to a well-defined interface. In practice, this usually implies a “one-size-fits-all” function signature: in our example, for candidate solution type Sol (e.g., a list of cities in the Traveling Salesperson Problem) perturbation is assumed to have type . However, suppose we now wish to incorporate a further heuristic that requires information about the search trajectory, for example a tabu list of solutions (Glover and Laguna, 1997) that prevents the resampling of those previously encountered. Not only must the configurator grammar be extended to include the possibility of different tabu mechanisms, but we must also change the implementation of local search to keep track of the trajectory. This clearly violates the OCP.
In Listing 2, we give an updated version in which a list of previous incumbent solutions is denoted by Sol.
The modified implementation now supports solution-based tabu mechanisms, but we will incur further violations of the OCP if we wish to incorporate components with new state dependencies, for example:
Metropolis-Hastings acceptance, which requires the delta of fitness values (Kirkpatrick et al., 1983).
Perturbation with a tabu operator (e.g., permutation indices) (Glover and Laguna, 1997).
“Simulated Annealing with reheating” (Luke, 2013), in which the acceptance criterion and the reheating procedure need access to the value of temperature.
Explicitly incorporating all of these features, merely because some component may require it, is computationally expensive and does not scale. More importantly, it simply cannot be anticipated in advance what information will be required by some component that is yet to be devised. In order to accommodate this, we need a generic way of providing extrinsic state, that is, (potentially shared) state that is maintained across component invocations, but which does not appear in the signature of component interfaces.
In the case of most existing metaheuristic implementations, extrinsic state is represented in an essentially ad hoc manner by passing references to shared state between components (e.g., via nonlocal variables). This is problematic for automated assembly, as explicit representation of these shared variables must be propagated through the call graph of the implementation, connecting components to the objects that they use. Furthermore, any variables that are declared in an ad hoc way throughout the implementation must be gathered manually in order to present them to a configuration tool. We now discuss our approach to solving this problem.
3 The Automated Open-Closed Principle
The “Open-Closed Principle” is usually ascribed to Meyer (1988) and states that frameworks should be “open” in the sense of being extensible via the addition of new components, but “closed” in that they do not require framework modification to achieve this.
The principle is typically expressed in terms of substitutability: the components are required to conform to established behaviors known a priori to the framework. The notion of component substitutability that suffices for conventional software development is that of “observable behavior,” typically defined by an interface that a component conforms to, along with “contracts” that specify pre- and post-conditions on component functionality (Liskov, 1987). A simple example of a component contract in metaheuristics is that the return value of accept: Sol × Sol → Sol should be equal to one of its arguments.
However, since new components may introduce new state dependencies, the OCP is insufficient to support flexible incremental assembly of metaheuristics from component parts. For example, an annealing mechanism that is used as a component in a local search algorithm may have a dependency on a temperature variable. In our example in Section 2, such introduction of new state dependencies requires manual propagation of the additional state through the call graph. We therefore propose the “Automated Open-Closed Principle” (AOCP) as an extension of the OCP:
A software framework should be open to automated configuration via combinatorial assembly over an extensible palette of components without requiring framework modification.
In order for a system's components to be highly reusable and substitutable, to facilitate incremental design, and thus also to be amenable to automated assembly without human intervention, a system must obey the AOCP. To conform to the AOCP, a system should exhibit the following properties:
Open: Present a framework parameterized by component definitions, together with an open-ended palette of such components.
Closed: Allow the addition of new components to the palette, without requiring human intervention to modify either the framework or pre-existing components.
Introspectable Design: Component signatures and dependencies on shared state are available in machine-readable form.
Observable: State transitions can be made fully observable to the assembly process.
To researchers who are accustomed to writing their desired metaheuristic from scratch, the proposed approach might initially appear to offer few benefits. However, it is most meaningful to consider the AOCP in the wider motivating context: that of large scale, potentially automated, assembly of metaheuristics for scientific discovery and reproducibility (Swan et al., 2015). The guiding notion is of a shared, community-wide repository of frameworks and components that can be freely combined to gain better insight into the correlation between problem features and solution mechanisms (Smith-Miles et al., 2014).
Without the proposed approach, exploring combinations of components at such a large scale would simply be impossible, due to the amount of manual intervention that would be required to manage state dependencies. An asymptotic cost of the associated manual labor can be obtained as follows: given a graph for a design with vertices (where vertices equivalently represent either grammar elements or subroutine invocations in the call graph), then for all pairs of components which are coupled via shared state, all paths between those components need to be manually updated to propagate that state. For each pair , then it is well-known,1 that the number of simple paths between u and v is given by . Since there are up to such pairs of vertices, the number of updates is given by .
4 Implementing the AOCP
We now outline an implementation of the ACOP, exploiting well-established programming constructs that directly support the ACOP requirements. This implementation uses functional programming techniques, since such techniques lend themselves well to clearly defining and managing state dependencies. The mechanisms that we describe can nonetheless be realized in an imperative manner. However, this requires work on the part of the framework implementer to achieve something that can already be achieved automatically by the means that we describe below. We are not suggesting that metaheuristics practitioners should become skilled functional programmers; rather the intention is that repositories of frameworks and components can be created using these principles and can subsequently be recombined by practitioners, via a layer of automation that hides the more technical detail.
Our implementation uses state threading as an alternative to the ad hoc approaches to extrinsic state taken by most contemporary metaheuristic implementations. State threading requires an explicit input/output parameter for the extrinsic state, which is then threaded through the search process. For example, in Listing 2, the extrinsic state for localSearch comprises the list of previous incumbent solutions, history. Instead of defining history as a shared or global variable that any of the three components access and update directly, perturb, accept, and finished each take an additional parameter that is used to thread the extrinsic state through the search algorithm.
Explicit representation of state facilitates the composition and reuse of components, possibly automatically, since all dependencies for that component are clearly described and can be easily reasoned about mechanically. Explicit representation of state also allows a component to dynamically inspect the environment that is passed to it. Such checks might be performed prior to the application of a perturbation, for example, to automatically adjust parameters and so improve its effectiveness. Explicit state and modularity enables static safety checks whilst also providing the means to concisely develop self-adjusting components, without creating an overly tight coupling.
Despite the advantages of this approach, manually threading the state through the algorithm, as in Listing 2, reduces the clarity of the code by introducing additional boilerplate. It is also error prone, since the programmer must ensure that the correct value of the state is passed to the correct stage of the algorithm. It is instead desirable to use some mechanism that implicitly threads the state, but does so in a well-defined and consistent manner.
The explicit representation of shared state in this manner is a well-known design pattern in the functional programming community, where it is expressed as the State monad. Whilst a formal definition is highly technical (Mac Lane, 1969; Moggi, 1991), it suffices here to consider monads to be a principled means of sequencing computations whilst abstracting over possible effects, such as state manipulation. Monads are a functional design pattern that follow well-defined laws to give strong formal guarantees to the programs that use them (Wadler, 1995). They provide a general abstraction mechanism that can be used to explicitly represent state; to clearly separate different kinds of state manipulation or other effects (such as I/O or exception handling), and to improve the reproducibility of behaviours on a per-component basis. A statically checked type-system can be used to separate stateless from stateful operations and to provide information about which state components are being manipulated.
Languages such as Haskell and Scala provide syntactic sugar for monads. In particular, they allow a sequence of flatMap operations to be chained together using syntax that looks like a traditional for loop. This is illustrated in Listing 3, a reformulation of our local search example in Listing 1 that uses the State monad. The state, in this case an integer representing the number of iterations, is implicitly threaded through each stage of the computation. This can be extended to yield a reformulation of Listing 2, thus incorporating history by changing the type of the state from Int to (Int, [Sol]) and modifying all components and state accesses, for example. Although the algorithm looks similar to its imperative counterpart, in our case all the state effects are explicit and the type system delineates precisely where they can occur: there are no implicit side effects and the validity of the sequence of operations can be checked by the compiler using its normal type checking mechanism.
In order to ensure that individual algorithms are easily and mechanically composable, we generalize over the type of the state. For example, instead of specifying that the type state has type Int in Listing 3, we use the type variable Env (“environment”). In principle, this abstracts over all possible types of environment, and results in a single generic definition of Local Search. Since we have abstracted over the environment, components still need a mechanism to access the information within the environment. The contemporary functional programming solution is to use lenses (Foster et al., 2005). Informally, a lens of type Lens[Source,Target] is a composable mechanism that focuses an object of type Source into an object of type Target that is contained within Source, for example by selecting a subfield of a record type. This is illustrated in Listing 4, a reformulation of Listing 3 to use lenses in addition to the State monad. Instead of explicitly accessing the environment in a manner that assumes a specific Env, the lens iter is used to express the same operation in a generic manner. The lens abstracts the access mechanism for a specific Source and Target of the algorithm or component.
Should the practitioners wish to add solution history to the local search algorithm in Listing 4, for example, they now only need to add a lens to access the solution history that is accessible from any components that may wish to access it. This is illustrated in Listing 5, which defines an acceptance component with access to the solution history via the lens history. Without the lens, reformulating Local Search in order to include solution history requires the practitioner to modify all components and state accesses within the algorithm itself.
In our approach, component dependencies are expressed via lenses in order to allow them to be as agnostic as possible about the specific means by which the Target value is obtained from the Source value. Lenses are used in conjunction with monads in order to allow for simple composition of components with explicit state dependencies. This ensures that all state dependencies are clearly defined and are available in a machine-readable form. Monads and lenses facilitate the AOCP as follows:
Open: State information explicitly manipulated by LocalSearch is provided in the signature of apply of Listing 4,5 to be made available for reusable components, for example, as described in Section 5. A subsequently authored component can get access to this state by providing an appropriate lens.
Closed: As described above, the key motivation for the use of State monads is precisely that they allow the framework to meet the extended closed requirements of the AOCP w.r.t. management of state dependencies.
Introspectable Design: The strong typing of the proposed approach means that the assembled system is guaranteed to have both state and parameter dependencies satisfied.
Observable: Since the State monad is responsible for propagating all state transitions, they can be directly observed. Further, this information can be made available as a form of dynamic instrumentation, to be exploited online by other components.
Related Work on State Handling
A number of authors have studied state-handling in metaheuristics via combinators—pure functional, self-contained components. Following initial work in genetic programming and exhaustive search (Briggs and O'Neill, 2008), monadic combinators were used in constraint programming (Schrijvers et al., 2013), with subsequent work in metaheuristics (Senington and Duke, 2013). The desirability of the monadic approach for metaheuristic design was recently advocated (Swan et al., 2015) as part of a wider research program, echoed by an emphatic call for reusable libraries such as CILib (Pampara and Engelbrecht, 2015). The only work that we are aware of in which state-handling is used in explicit support of metaheuristic assembly is given by Kocsis et al. (2015), who propose a combinator-based approach where component dependencies are amalgamated into a shared workspace.
5 Example Application of the Automated Open-Closed Principle
In this section, we describe a system that follows the AOCP. Here, as discussed in the previous section, we use a combination of State monads and lenses to define a local search framework and its component parts. Subsequently, we expose these components to an automated tool, which assembles them into a working local search algorithm. As our target problem domain, we have chosen the well-known Traveling Salesperson Problem (TSP) (Luke, 2013). The experiments performed in this section are merely illustrative; that is, they aim to demonstrate that the resulting local search framework indeed supports “automated combinatorial assembly” as is intrinsic to the AOCP. It is not our objective to design a heuristic competitive with the state of the art for the TSP, since the additional complexity of the more sophisticated components required would obfuscate the key points we wish to convey.
As per Listing 4, we consider a local search framework that has three top-level design choices (perturb, accept, isFinished). We consider a single termination condition (isFinished), terminating local search after iterations. We consider the following types of perturbations (perturb):
RandomSwap: Swaps two cities at random.
RandomInsert: Removes a randomly selected city, reinserting it in a random position.
RandomShuffle: Randomly exchanges all cities.
RandomShuffleSubset(): Selects cities at random and exchanges them randomly, where n is the number of cities in the TSP instance.
ReverseSubtours(): Reverses randomly selected subtours.
Remark that the last two perturbations are in turn parametrized, taking a real-valued mutation rate as argument. We also consider three types of acceptance conditions (accept):
AcceptImproving: Accepts all incoming tours shorter than the incumbent.
AcceptImprovingOrEqual: Accepts all incoming tours no longer than the incumbent.
AcceptMetropolisHastings(): Accepts incoming tours with likelikhood , where and are the length of the incumbent and incoming tour respectively, and T is the current temperature ( a positive scalar). The initial temperature is instance-specific and determined as described by White (1984). The value of T in subsequent iterations is dynamically controlled by a cooling schedule s. We consider two types:
– LinearCoolingSchedule: The temperature after iterations is ).
– GeometricCoolingSchedule(): Reduces the current temperature with a factor r.
These components have a number of different state dependencies. The State monad in our example keeps track of:
iter: The number of iterations performed.
maxIter: The optimization budget.
T: The current temperature.
rng: A stream of (pseudo) random numbers.
f: A procedure for computing the quality of a solution (length of a tour in our example).
These component-specific state dependencies are modeled using lenses, as shown in Listing 6. Notice that each component is agnostic about the environmental dependencies of any other component, resulting in loose coupling. Most significantly regarding the AOCP, the core LocalSearch framework can therefore remain unchanged regardless of the complexity of the components with which it is configured. Whilst this framework is straightforward to implement, algorithms such as Scatter-search or Learning Classifier Systems (Luke, 2013) are notoriously difficult to implement correctly; the ability to define a truly reusable reference implementation via the AOCP serves a vital scientific purpose in ensuring the validity and reproducibility of reported results.
Using this collection of components, a set of different candidate local search methods can be assembled. Remark that this set (a.k.a. the design space) is infinite, as the parameters m and r can take any real value in [0,1]. However, in this illustration, we consider only the finite subset of 350 candidate designs with .
In automated assembly, the objective is to (automatically) determine which combination of components is best suited for solving a given target problem (e.g., the TSP). In literature, a variety of different approaches have been explored to do so. A prominent example is genetic programming (Koza, 1994), which solves this (meta-)optimization problem using evolutionary algorithms. As the focus of this article is the AOCP itself, we do not advocate any specific optimizer—rather, we illustrate two different approaches:
Ant Programming (AP; Boryczka, 2002) is a variant of Ant-Colony Optimization (Luke, 2013) in which the combinatorial structure to be optimized is a program tree. As our optimizer, we use ContainAnt (Kocsis and Swan, 2017), an AP variant implemented in Scala.2 The search space of structures can usefully be described via a grammar. In the case of ContainAnt, this grammar is specified directly in Scala code, by the fields and methods of a user-defined subclass of containant.Module. By this means, it is possible to specify any context-free grammar in terms of the attributes (val) and method signatures (def) that are automatically obtained from the grammar module via reflection. Notice that this grammar is statically typed and this can also be considered as a form of embedded Automatic Improvement Programming (Kocsis and Swan, 2018). The grammar for our example is shown is Listing 6. Note that ContainAnt was not designed with any explicit knowledge of monads and lenses: to the Ant Programming search algorithm, they are treated just like any other type.
Programming by Optimization (PbO; Hoos, 2012). In PbO, design choices are exposed as program parameters whose values correspond to alternative decisions. Subsequently, the best combination of parameter values (configuration) is determined automatically with the help of automated tools known as algorithm configurators. We used SMAC (Hutter et al., 2011b) as configurator in our illustration.3 We exposed our design choices in the form of 5 parameters: perturb, m, accept, s, and r; having 5, 11, 3, 2, and 11 possible values, respectively. As each component is parametrized by its design choices, doing so did not require us to modify any existing code. Nonetheless, creating the “tuning scenario” (wrapper program, parameter specification file, etc.) was a tedious and error-prone process. To alleviate this issue, language extensions have been proposed in prior art (Hoos, 2012; Ansel et al., 2009), supporting the manual “annotation” of design choices in code, allowing them to be extracted automatically. Interestingly, in a system following the AOCP this process could be automated using reflection, supporting statically typed specifications in native code similar to Listing 6.
Both AP and PbO evaluate “how good” a candidate design is by testing it on instances of the target problem (a.k.a. training instances). Notice that, to avoid bias, a disjoint set of instances must be used to assess the performance of the design returned by the optimizer (a.k.a. test instances). The TSP instances that we consider here are the subset of 76 symmetric TSPs from the well-known TSPLib that have edge weights in the “Euclidean 2D format.”4 More specifically, the training set consists of the 12 instances with 100 cities or fewer and the test set consists of the 57 (of 64) remaining instances with fewer than 4,000 cities. It is well-known that automated algorithm design is highly computationally intensive, and these smaller instances are used so that all experiments can be completed in under 72 hours on a modern desktop PC.5
We run both ContainAnt and SMAC once, using their default parameter settings, permitting them a total of 6000 tests to determine which of the 350 candidate local searches minimizes the average Relative Error (RE), given by , on the training instances, when applied to an initial solution obtained by the nearest neighbor heuristic. Notice that both ContainAnt and SMAC are anytime; that is, they can at any time be queried for the best design they found thus far. Figures 1 and 2 show the average RE on the training and test instances, respectively, of the anytime solution obtained by each meta-optimizer.6
We observe that both SMAC and ContainAnt discover better designs (lower average RE) over time. In this pair of runs, ContainAnt clearly started from a worse initial design. However, after 48 tests it identified a better alternative and in the remainder of its run found equally good designs roughly 2–3 times faster than SMAC. While this result provides additional evidence of the competitiveness of ContainAnt (c.f. Kocsis and Swan, 2017), it is insufficient to draw any conclusions about the relative performance of SMAC and ContainAnt in general. In other tuning scenarios (or another run) SMAC may do better. Also note that SMAC has various features (e.g., adaptive capping, incremental evaluation) which will likely make it preferable in certain settings (e.g., optimization of runtime, noisy fitness criterion).
After 6000 tests, both SMAC and ContainAnt return the same design (using ReverseSubTours(0.2) and AcceptImproving), suggesting it is the best (on the training set) of the 350 candidates considered. This design obtains an average RE of 0.061 on the training instances, and an average RE of 0.146 on the test instances. Notice that the higher average RE on the test instances is likely due to the fact that these instances are harder. Given the similarity of Figures 1 and 2, we do not believe our results suffer from over-tuning. For completeness, Figure 3 shows the average RE of this design on each of the 57 test instances. Unsurprisingly, its performance is not competitive with that of the state of the art.
6 Consequences of the Automated Open-Closed Principle
As well as addressing the immediate problem of state dependencies, the mechanisms of the AOCP bring many benefits to metaheuristic assembly. Here, we discuss the most significant.
By explicitly expressing state dependencies, following the AOCP facilitates the creation of parallelizable metaheuristics. While mechanisms such as monads express the logical ordering of operations, they do not enforce sequential execution where dependencies do not exist. These facts can be exploited to automatically introduce parallelism (Castro et al., 2016; Scaife et al., 2006), and to restructure programs to automatically expose alternative parallelizations, so allowing automated determination of the most efficient parallel program. Structured parallelism techniques, such as algorithmic skeletons (Cole, 1988), can then be used to obtain a good parallel implementation. Such techniques take a high-level, modular approach to parallelism, presenting to the programmer high-level patterns that abstract low-level implementation details such as communication, task creation, and scheduling. They avoid a number of problems such as race conditions and deadlocks that are difficult to diagnose or debug (Barwell et al., 2016).
Structured parallelism techniques have a long-observed correspondence with functional programming. For example, the “Par monad” library for Haskell (Marlow et al., 2011) provides the parMapevaluation strategy (Trinder et al., 1998), that applies a given function to each element in a data-structure in parallel in a simple and thread-safe way (Hammond and Michaelson, 1999). To illustrate this, consider the Evolutionary Multi-Agent System (EMAS) given in Listing 7. EMASs are a hybrid metaheuristic that combine multi-agent systems with evolutionary algorithms. Each agent represents a solution to a particular optimization problem that is iteratively improved by a series of reproductions and fitness contests with other agents (Stypka et al., 2018). The population of agents is subdivided into islands. Randomized migration between islands maintains diversity. Agents cooperate with or compete against other agents on the same island, and are periodically chosen for migration to other islands. During this process, meetingAgent is applied to each island of agents as a map operation, and is parallelized using parMap, where runPar enables the use of parMap.
6.2 Credit Assignment
Various mechanisms (e.g., perturbation scheme, acceptance, and restart condition) affect the behavior of a metaheuristic. This gives rise to a structural Credit Assignment Problem (CAP; Minsky, 1961): Which of these mechanisms is to blame for/contributes to the poor/good performance we observe on a particular problem instance?
Beyond its scientific merit, the information that is available from credit assignment can be exploited to guide automated metaheuristic assembly. For instance, ParamILS (Hutter et al., 2009) performs an (iterated) local search in the one-exchange neighborhood; that is, it changes a single component at the time and figures out whether this improves or worsens the performance of the incumbent design and updates it accordingly. SMAC (Hutter et al., 2011a) builds a regression model mapping configurations to performance predictions, effectively learning which combinations of parameter values perform well together. SMAC uses this model to determine which configuration to try next.
These approaches share the property that credit is assigned post-execution, based on correlations between the performance observations of the system as a whole and its components. While the AOCP supports these correlation-based approaches, it also facilitates the study of causality (Adriaensen and Nowé, 2016b) by examining the behavior of components during execution (c.f. previous work relating execution state to evaluation; Krawiec and Swan, 2013). This allows for the decoupling of design and performance spaces, enabling performance observations to generalize beyond the specific design that is used to obtain them, based on similarities in execution space (e.g., using Importance Sampling; Adriaensen et al., 2017). This has the potential for more efficient, accurate, and insight-promoting credit assignment.
A simple example of behavioral information that can be exploited to perform more accurate credit assignment is unused parameters. Often, not all parameters of a metaheuristic framework are used during every run. The values of these unused parameters can obviously not be held responsible for the performance observed. Remark that while some contemporary configurators (e.g., ParamILS and SMAC) support parameter dependencies (a.k.a. conditional parameters), they are oblivious of the fact that even “active” parameters are sometimes not used (e.g., for certain problem instance / random seed combinations). Falsely assigning credit to the values of unused parameters introduces unnecessary noise, complicating configuration.
More generally, when taking a behavioral perspective, the structural CAP reduces to a temporal CAP, a problem that is widely studied in the Reinforcement Learning community (Sutton and Barto, 1998). Of particular interest to the modular approach laid out in this article is “hierarchical” reinforcement learning (Barto and Mahadevan, 2003), which studies the problem of learning complex tasks through learning simpler sub-tasks and exploiting their interdependencies. Also, the temporal CAP can be solved online using Temporal Difference (TD) techniques (Tesauro, 1995) such as Q-learning (Watkins and Dayan, 1992). Such methods are able to learn what works best and to exploit this knowledge while solving the problem. While the use of these techniques in the context of metaheuristics has been explored (e.g., in the area of reactive search; Battiti et al., 2008), their full potential is arguably yet to be unlocked, and it is our belief that the AOCP will also greatly facilitate this endeavor.
The intrinsically recursive nature of metaheuristics has been widely noted (Vaessens et al., 1998; Swan et al., 2011, 2014; López-Ibáñez et al., 2014). Using a truly modular approach, combinations of components can themselves be considered as components (Woodward et al., 2014). For example, the local search framework of Listing 4 can itself be treated as a perturbation operator. The AOCP supports the full exploration of this recursion, enabling the exploration of deeply nested combinations of heuristics.
6.4 Escaping the Poverty of Metaphor
Following the success of Genetic Algorithms, many metaheuristics have been derived via “reasoning by metaphor.” This has led to a very large number of competing metaheuristics, and the relationship between them is often unclear, a situation that has recently been criticized (Sörensen, 2013). Without true separation of concerns, it is difficult to either formally identify the equivalence of two components or to combine components in an automated manner, making it impossible in many cases for one metaheuristic to borrow components from another. A modular representation escapes these limitations and offers opportunities to explore a much greater design space via automated hybridization.
As noted in Section 6.2, the absence of hidden state allows unused parameters to be discovered in order to properly identify useful components. An extreme example of the importance of progressing beyond the current situation is given by Nair et al. (2016), where a superior algorithm that did not incorporate an evolutionary metaphor was discovered by accidentally disabling an evolutionary component. The proposed approach provides the opportunity to discover such algorithms automatically, allowing elimination of accidental complexity to become routine practice (Adriaensen and Nowé, 2016a).
7 Conclusions and Future Work
State-of-the-art approaches to automated metaheuristic configuration and design require considerable “human-in-the-loop” intervention to extend the frameworks they provide. Manual effort is required to manage interdependencies between the framework and its components. To address this problem, we introduce the AOCP, an extension to the well-known “Open-Closed Principle” of software engineering, which we claim is essential to scalable design automation not only of metaheuristics, but of algorithms in general. Adoption of the AOCP leads to design space descriptions that:
Are reusable at the level of both frameworks and components.
Are purely functional, with unintrusive support for component-specific state.
Can be assembled “bottom-up,” from an open-ended palette of components.
Do not require scale-dependent human intervention when incorporating components with new dependencies.
Future work will explore the potential for scalability and reusability across a range of alternative metaheuristic frameworks, both in terms of the size of the palette of components and the size of the problems to be addressed. Since the notion of the AOCP that we propose is a general one, it can further be used for automated assembly in other domains, including machine learning and software engineering.
The utility of communal design templates forms a cornerstone of the “Metaheuristics in the Large” community initiative (Swan et al., 2015). The authors would like to thank all the many collaborators in this initiative.
Strictly, for .
https://github.com/zaklogician/ContainAnt. (source code)
http://www.cs.ubc.ca/labs/beta/Projects/SMAC/ (version 2.10)
Specifically, Windows 10, Intel™ i5 CPU 3.4 GHz.