A Hidden Markov Model Approach to the Problem of Heuristic Selection in Hyper-Heuristics with a Case Study in High School Timetabling Problems

The field of search and optimisation in operations research has a long and varied history. Many methods have been developed that solve specific problems well and rely on problem-specific knowledge to function. Although highly computationally efficient, many of these methods are necessarily specific to each problem and so cannot easily be applied to other problems without significant modification. The research community has therefore looked to metaheuristics as generic problem-solving methods, for example, genetic algorithms (Holland, 1975), tabu search methods (Glover, 1986), and other nature-inspired methods such as simulated annealing (Kirkpatrick et al., 1983) are widely used. However, these methods usually require significant expertise to implement and tune for specific problems, and in their standard versions at least, are unable to adapt to changing search spaces. Despite the efforts of the scientific community in developing a variety of search methodologies, there is no known efficient method that offers the best performance for solving different problems and for all possible situations, and in fact it has been proved that this is unachievable (Wolpert and Macready, 1997). Despite this, there has been scientific progress in designing general methods which are valid for a wide range of, rather than all, problem domains. Such systems are applicable to different instances from not only the same domain but also other problem domains. Moreover, they are easy to build and maintain and so are less costly in terms of the investment in time required to apply them to new problems. Hyper-heuristics have emerged as such general purpose, high-level search methodologies which are motivated by the goal of selecting or generating heuristics automatically to solve a wide range of difficult optimisation problems (Burke et al., 2013). This work focuses on the selection type of hyper-heuristics.

Traditional selection hyper-heuristics have focused on the improvement that single or paired low-level heuristics can bring to an optimisation. The task for these methods is to select the most appropriate and best performing heuristic for a given point in the optimisation. A wide number of methods have been proposed for this task including the simple random hyper-heuristic, choice function hyper-heuristic, and other methods presented in Burke et al. (2013). However, in scientific fields where sequences are known to be prevalent, for example in bioinformatics and language processing, a key feature is that context is found to be important and that the “meaning” of a token in the sequence cannot be determined unless the context in which it finds itself is also considered. Here it is proposed that it is possible to extend this context principle to the selection of heuristics within the search domain, noting that the effectiveness of search operations is determined to a certain extent by those that have been executed before it. In this article, we extend this principle to investigate the potential for the analysis of sequences of operations in search and optimisation problems to construct building blocks of good heuristic combinations.

This work studies and analyses the performance of two sequence-based methods, a simple fixed parametrised method and a hidden Markov model (HMM) approach (Kheiri and Keedwell, 2015b; Kheiri et al., 2015). A hyper-heuristic with a fixed parametrised sequence size is implemented to allow for experimentation on the sequence lengths and to discover information regarding the information that can be gained from each such sequence. In addition, the HMM method is shown to learn online the optimum sequence lengths automatically and is able to adapt the probability of heuristic application and sequence-based acceptance strategy to the search landscape. Experimentation with these methods is conducted on difficult problems from timetabling, a key problem area in operational research.

The high school timetabling problem is a hard combinatorial optimisation problem (Even et al., 1976). A solution requires the scheduling of events and resources in time slots subject to a set of hard and soft constraints. A solution is expected to satisfy all the hard constraints and as many of the soft constraints as possible. The importance of the high school timetabling problem stems from its difficulty due to the number of constraints involved, the NP nature of the problem (Even et al., 1976), and the need to perform this hard task by educational institutions everywhere; thus it became necessary to search for methods that help with automating this process.

The article is structured as follows. Section 2 overviews selection hyper-heuristics and high school timetabling problems. Section 3 describes the high school timetabling benchmark used for the experimentation. Section 4 describes the developed method including the algorithmic components and low-level heuristics. In Section 5, the performance of the developed methods is analysed and compared against the state-of-the-art approaches. Section 6 provides conclusions and areas for further work.

The term “hyper-heuristic” was used by Cowling et al. (2001) to describe a high-level problem-independent method that provides a solution methodology to solve a wide range of optimisation algorithms. Hyper-heuristics can be broadly classified into selection hyper-heuristics to select from a set of predefined heuristics, and generation hyper-heuristics which differ from selection methods in that they generate heuristics (Burke et al., 2013). This work focuses on the former class of hyper-heuristics.

Selection hyper-heuristics are motivated by the understanding that each heuristic performs differently on different problem instances and an approach that combines them could yield better overall performance (Burke et al., 2013). An iterated selection hyper-heuristic method (Özcan et al., 2008) aims to improve the current solution by selecting and applying a low-level heuristic from a set of predefined low-level heuristics (e.g., mutation operators and local search heuristics), leading to a new solution; then the move acceptance method decides whether to accept or reject the modified solution. This cycle of applying selection and move acceptance methods is repeated until a termination criterion is satisfied. The number of studies on selection hyper-heuristics is growing massively (Burke et al., 2013) and as such, space prevents us reviewing all such approaches. However, we will overview the methods that are relevant to those techniques proposed in this study.

The heuristic selection method is described in Section 4 but this must be coupled with a move acceptance method to determine whether to select or reject the generated solutions. A set of well-known metaheuristic-inspired move acceptance methods, including hill climbing (only improving) (HC), simulated annealing (SA), great deluge (GD), record-to-record travel (RR), and late acceptance (LA) methods are used as move acceptance criteria in this study.

A deterministic move acceptance method which accepts only improved solutions is described by Cowling et al. (2001). We refer to this method as the hill climbing move acceptance method.

Dueck (1993) proposed another variant of GD named the record-to-record travel (RR) method. The idea of RR is based on the simple notion that any new solution, which is not much worse than the best solution recorded, is accepted.

The late acceptance method (Özcan et al., 2009) is a variant of the hill climbing method. A candidate solution in the late acceptance method is accepted if its quality is better than a solution which was obtained L steps before. The method requires an implementation of a circular queue of size L which maintains the cost values of L previously visited solutions.

Due to the extreme difficulty of high school timetabling problems, metaheuristics are preferred in most of the studies. Abramson (1991) described the high school timetabling problem and then proposed a simulated annealing solution method. The author presented a parallel algorithm and proved experimentally that the developed method works faster than the equivalent sequential algorithm. The tests were made on data from an Australian school. The THOR school timetabling tool was developed by Melício et al. (2006) for Portuguese schools. An initial solution is first created using a heuristic construction algorithm, which is improved further by applying fast simulated annealing. The tool was used by more than 100 Portuguese schools and was applied with great success. Zhang et al. (2010) used a simulated annealing–based algorithm with a newly designed neighbourhood structure to solve the high school timetabling problem.

One of the earliest studies that applied the tabu search to solve the high school timetabling problem was suggested by Wright (1996). The conducted study produced a timetable for a large comprehensive school in England, using a solution method that involved heuristic search and a form of tabu search. Alvarez-Valdés et al. (2002) developed an algorithm based on tabu search for teacher assignment in Spanish secondary schools. The algorithm was implemented in three phases. In the first phase a parallel heuristic algorithm is used to build an initial solution, and in the second phase the solution is improved using the tabu search to obtain feasible solutions. In the third phase the solution is improved further. The algorithm was applied to 12 Spanish school instances and proved to provide better allocations than the manual ones. Jacobsen et al. (2006) developed an approach that generates an initial solution using a construction heuristic with a graph colouring algorithm. The solution is then improved using tabu search. The algorithm was tested on data from German high schools. Bello et al. (2008) treated the high school timetabling problem as a graph colouring problem, and used it in association with the tabu search algorithm. The system was applied to instances from Brazilian high schools.

Colorni et al. (1992) used several metaheuristics based on genetic algorithms (GA), simulated annealing (SA), and tabu search (TS) and compared them using instances of Italian high school data. The research results showed that a hybrid of a genetic algorithm with local search could give promising performance. Calderia and Ross (1997) evaluated the use of genetic algorithms to solve instances of school timetabling that are randomly generated. An initial population of feasible timetables are produced in an initial procedure and the GA is used to improve the quality of the generated population. A highly constrained school timetabling problem extracted from the requirements of a German high school was investigated by Bufé et al. (2001) using a hybrid approach. An evolutionary algorithm combined with local search that uses specific mutation operators to optimise the given timetables was used to find feasible solutions. Filho et al. (2001) used a new representation for the high school timetabling problem by forming clusters from pairs of teachers and classes. The authors applied a constructive genetic algorithm to solve instances of two Brazilian high schools. Wilke et al. (2002) presented a genetic algorithm for solving the German high school timetabling problem. A hybrid genetic approach is applied using multiple genetic operators which proved to perform better than the traditional genetic algorithm. Beligiannis et al. (2008) solved the high school timetabling problem using an adaptive evolutionary algorithm. The algorithm did not employ a crossover operator, and the results showed the success of the approach when applied on the Greek high school timetabling problem. Raghavjee and Pillay (2008) applied a genetic algorithm to the school timetabling problem. The algorithm is based on creating an initial population of timetables and then applying a mutation operator to refine this population. It was tested with five high school timetabling problems and proved to generate better results than all the methods tested with the same set. Raghavjee and Pillay (2012) compared the performance of a genetic algorithm and genetic programming using the Abramson (1991) dataset of five high school timetabling problems. The genetic programming approach proved to give a better performance in the five problems over other approaches including genetic algorithm, neural networks, tabu search, and greedy search, especially in the large problems in the set.

Other approaches used in the high school timetabling problem include adaptive large neighbourhood search (Sørensen et al., 2012), integer programming (Birbas et al., 2009), particle swarm optimisation (Tassopoulos and Beligiannis, 2012), tiling algorithms (Kingston, 2005), bee algorithms (Lara et al., 2008), Hopfield neural networks (Smith et al., 2003), walk down jump up algorithm (Wilke and Killer, 2010), greedy randomised adaptive search procedure (Moura and Scaraficci, 2010), and constraint programming approach (Valouxis and Housos, 2003). For the survey on the high school timetabling problem, the reader is directed to  Pillay (2013).

Due to the different education systems and constraints imposed by different educational institutions, a group of researchers (Post et al., 2012) has proposed a unified high school timetabling format that fits with the different education systems across the world. To encourage scientists and practitioners to provide solution methods for the high school timetabling problem, an international timetabling competition (ITC 2011)¹ (Post et al., 2013) using the unified benchmark instances collected from over ten countries was organised.

The competition consisted of three rounds. In the first and third rounds, competitors were expected to submit solutions to a set of instances without any restriction on the resources or techniques used to generate these solutions. In the second round, solvers were submitted to the organisers and tested on a set of instances in a specified time limit of 1000 seconds.

The problem instance consists of times which are the intervals of time in which events run; resources which are the entities that attend events; and events which specify the coordination of resources. Solutions to the timetabling problem consist of the allocation of resources to events. The resources that must be allocated are the group of students (known as a class), the teacher, and the room in which the event will take place. Each resource has a set of constraints associated with it (e.g., limitations on the number of classes students are required to take in one day, teacher workload, and room capacities). Events can be single lessons or a set of lessons (an event group) and each event has a number of properties that influence the allocation of resources to them through constraints or other optimality criteria. These are the event duration, pre-assigned resources (e.g., some events must take place in certain rooms), the contribution of the event to workload, and pre-assigned time slots. Fifteen hard/soft constraints are expected to be satisfied (Post et al., 2013):

• C01 Assign resource: Assign resource to event.

• C02 Assign time: Assign time to event.

• C03 Split events: Split event into subevents under specific constraints.

• C04 Distribute split events: Split event into subevents under constrained durations.

• C05 Prefer resources: Assign specific resource(s) to event.

• C06 Prefer times: Assign specific time(s) to event.

• C07 Avoid split assignments: Assign the same resource to a set of events.

• C08 Spread events: Spread events evenly through the cycle.

• C09 Link events: Assign the same time to a set of events.

• C10 Avoid clashes: Assign resources without having clashes.

• C11 Avoid unavailable times: Avoid assigning resources at unavailable times.

• C12 Limit idle times: Avoid having idle times for resources.

• C13 Cluster busy times: For a number of days, resources must be busy.

• C14 Limit busy times: For a number of times, resources must be busy.

• C15 Limit workload: Schedule the workload without exceeding a limit.

The quality of a solution is evaluated in terms of (hardViolationScore,softViolationScore). A solution with a cost of (25,78) indicates an infeasibility value of 25 (sum of weighted hard constraints violations) and objective value of 78 (sum of weighted soft constraints violations). The weight value and whether the constraint is hard or soft per each constraint type are presented in the instance.

The instances used during ITC 2011 are still available online in the competition website, but deprecated. The developers of the benchmark project suggested the focus on solving a set of instances, referred to as XHSTT-2014 instances, which contains a carefully selected set of worldwide instances in their most up-to-date form. However, in this work, we used the ITC 2011 instances of round 2 and compared the performance of our approach against the ITC 2011 solvers using the same rules imposed for the second round of the competition. We then applied the developed method on XHSTT-2014 instances. Table 1 summarises the main characteristics of the ITC 2011 and XHSTT-2014 instances obtained from 12 countries.

Due to space restrictions, we provide only a brief description of the studied problem. For a full description of the problem, the reader is directed to Post et al. (2012, 2013).

Four competitors (GOAL, HySST, Lectio, and HFT) submitted solvers to the second round of the ITC 2011 competition (Post et al., 2013; Kheiri, 2014). GOAL combined iterated local search with simulated annealing (Fonseca et al., 2014). HySST applied a stochastic local search hyper-heuristic (Kheiri et al., 2016). Lectio employed an approach based on adaptive large neighbourhood search (Sørensen et al., 2012). HFT developed an evolutionary algorithm as a solution method (Domrös and Homberger, 2012). The ranking method was based on the average rank of ten independent trials with ten seeds chosen at random over eighteen selected instances per team with each run for 1,000 seconds. The GOAL team obtained the lowest mean rank and were therefore deemed the winner of the second round of ITC 2011. Soon after the competition, the results of the GOAL team were improved using a late acceptance hill-climbing method as reported in Fonseca et al. (2015).

Hyper-heuristic methods operate above the level of heuristics and so do not deal directly with the problem representation; the low-level heuristics provided are designed to work at this level. As such, the proposed hyper-heuristic algorithm works simply by invoking the KHE platform, an open source software tool written by Jeff Kingston (2014)² which constructs initial solutions using the concept of hierarchical timetabling utilising a tree structure to represent resource allocations to events. The exact implementation of the solution is not of concern from a hyper-heuristic perspective, but the interested reader is directed to Kingston (2014). However, an initial produced solution of a given problem instance generally violates many of the problem constraints (see Kingston, 2014). The proposed hyper-heuristic is, therefore, used to fix as many of these violated constraints as possible.

Recall that hyper-heuristics are composed of two components: the selection component and the move acceptance component. In the selection component, a low-level heuristic will be selected and applied to a current solution (S_current) and that will generate a new solution (S_new). Now, given S_current and S_new, the move acceptance will decide whether to accept or reject S_new. In this work, a sequence-based selection (SS) method described in Section 4.1 is used to select and apply sequences of heuristics. Five different reusable metaheuristics inspired methods are used as move acceptance including hill climbing local search (HC), simulated annealing (SA), great deluge (GD), record-to-record travel (RR), and late acceptance (LA) methods. For example, if we used HC as a move acceptance, then S_new will be accepted only if its quality is better than S_current; otherwise, it will be rejected. If we used SA, then S_new will be accepted if its quality is better than S_current or it will be accepted with a given probability.

Most of the parameters involved with the move acceptance methods are set to values which were suggested in previous studies. In the case of solutions with hard constraint violations, the F value in both simulated annealing and great deluge criteria is assigned to 0.01% of the cost of the best recorded solution during the search process; otherwise, the value is set to 1% of the cost of the best recorded solution in hand. Similarly, the value of f in GD is set to 0.001% of the cost of the best solution in hand, and to 0.1% if the best solution violates only soft constraints. These settings are suggested in Kalender et al. (2013) and Ahmed et al. (2015). The memory size L of the LA acceptance method is set to 500 as suggested in Özcan et al. (2009). The RR accepts the nonimproving solution if its cost is not worse than the cost of the best recorded solution in hand plus 0,5. This parameter is chosen after light experimentation.

The sequence-based selection hyper-heuristic approach in this study manages a set of 15 low-level heuristics to improve the quality of a single solution:

• LLH0: swap the time slots of two randomly selected events. As an example, assume that a Geography class event is assigned to the second time slot on Monday and the History class event is assigned to the third time slot on Friday. LLH0 will assign History class to the second time slot on Monday, and Geography class to the third time slot on Friday.

• LLH1: select two random events and swap their time slots in case they have the same duration or not adjacent; otherwise, the swap occurs but the first event is moved to follow the last time slot occupied by the second event. As an example, assume that a Geography class event with a duration of one is assigned to the first time slot on Monday and the History class event with a duration of two is assigned to the second time slot on Monday. LLH1 will assign Geography class to the third time slot on Monday (not to the second time slot on Monday), and History class to the first time slot on Monday. However, if both events have the same duration, then LLH0 will be invoked.

• LLH2: randomly select an event and reschedule to a random time slot. As an example, assume that a Mathematics class event is assigned to the first time slot on Monday. LLH2 will select a new random time slot, for example, the last time slot on Tuesday, and then it will unassign Mathematics from the first time slot on Monday and then assign it to the last time slot on Tuesday.

• LLH3: randomly select an unassigned event and then assign it to a random time slot. This low-level heuristic works similarly to LLH2; however, the event is expected to be unassigned to begin with. As an example, assume that a Mathematics class event is unassigned to any time slot. LLH3 will select a random time slot, for example, last time slot on Tuesday, and then it will assign Mathematics to the last time slot on Tuesday.

• LLH4: randomly select an assigned event and then unassign it. This low-level heuristic is the opposite of LLH3.

• LLH5: randomly assign and unassign several events. This is a ruin and recreate low-level heuristic which has a parameter that takes a value between 1 and 10. This parameter will be selected randomly each time this heuristic is invoked. The parameter represents the number of events to be assigned or unassigned. If, for example, the parameter has a value of 5, then LLH5 will select five events at random and then at each selected event the heuristic either applies LLH2, LLH3, or LLH4 (selected randomly with an equal distribution).

• LLH6: shuffle the assignment of several events. This heuristic has a parameter that takes a value between 1 and 10. This parameter will be selected randomly each time this heuristic is invoked. The parameter represents the number of events to be shuffled. If, for example, the parameter has a value of 5, then LLH6 will select five events at random and then shuffle their assignments.

• LLH7: split a randomly selected event into two events. This heuristic divides a randomly chosen event if it has an assignment of a time block of at least two consecutive time slots into two events such that their durations should sum to the duration of the original event. As an example, assume that a Geography class event with a duration of three is assigned to the first time slot on Monday; LLH7 will divide the teaching of Geography into two separate (still consecutive) time slots with one event having a duration of one and the other event having a duration of two. This low-level heuristic will allow for future moves to operate on those two events separately.

• LLH8: merge two randomly selected events adjacent in time and sharing the same events. This heuristic is the opposite of LLH7.

• LLH9: swap two random resources. As an example, assume that a Geography class event is assigned to Class Room A and the History class event is assigned to Class Room B. LLH9 will assign a History class to Class Room A, and a Geography class to Class Room B.

• LLH10: reschedule a resource element of an event. As an example, assume that Class Room A is assigned to a Geography class event. LLH10 could re-assign Class Room A to a History class.

• LLH11: randomly select an unassigned resource and then assign it at random. This low-level heuristic works similarly to LLH10; however, the resource is expected to be unassigned to begin with. As an example, assume that Teacher A is unassigned to any class. LLH11 will select a random class event and then it will assign Teacher A to the selected class.

• LLH12: randomly select an assigned resource and then unassign it. This low-level heuristic is the opposite of LLH11. As an example, assume that Teacher A is assigned to a given class. LLH12 will unassign Teacher A to the given class.

• LLH13: randomly assign and unassign several resources. This is a ruin and recreate low-level heuristic which has a parameter that takes a value between 1 and 10. This parameter will be selected randomly each time this heuristic is invoked. The parameter represents the number of resources to be assigned or unassigned. If, for example, the parameter has a value of 5, then LLH13 will select five resources at random and then at each selected resource the heuristic either applies LLH10, LLH11, or LLH12 (selected randomly with an equal distribution).

• LLH14: shuffle the assignment of several resources. This heuristic has a parameter that takes a value between 1 and 10. This parameter will be selected randomly each time this heuristic is invoked. The parameter represents the number of resources to be shuffled. If, for example, the parameter has a value of 5, then LLH14 will select five resources at random and then shuffle their assignments.

The hyper-heuristic implemented in this study aims to select and apply sequences of heuristics instead of selecting and applying a single heuristic. This section introduces a modified framework for selection hyper-heuristics enabling the operation of selecting sequences of heuristics. Figure 1 illustrates how a generic sequence-based selection hyper-heuristic framework operates.

This study applies a hidden Markov model (Baum and Petrie, 1966) approach for the sequence-based selection method (see Figure 2). The goal is to learn and generate sequences of low-level heuristics where low-level heuristics represent hidden states of the model. Each state (low-level heuristic) has a transition probability to move to another state (or itself) and a sequence-based acceptance strategy probability to decide whether a sequence of heuristics is constructed. Therefore we define two matrices: one to store the scores, hence the probabilities, to move from a heuristic to another; and the other to store the scores of the acceptance strategy for each low-level heuristic. We refer to the first matrix as and the second as ⁠. Note that the Markov chain model employed in the work in Kheiri and Keedwell (2015a) is a simpler Markov model where the state is directly visible to the observer and therefore the state (low-level heuristic) transition probabilities are the only parameters.

We provide an example of how the developed method would work on five low-level heuristics. Figure 3 shows the initial score values of the two HMM matrices. For example, if llh₂ is initially selected as the current low-level heuristic and considering the scores in the transition matrix, the probability to move from llh₂ to any other heuristic is 1/5. The hyper-heuristic applies the roulette wheel selection method to select the next low-level heuristic. We assume that llh₁ is selected next. The hyper-heuristic will then select the sequence-based acceptance strategy of llh₁. The probability at this point of selecting any of the two acceptance strategies is 1/2. We assume that the hyper-heuristic selects the acceptance strategy ⁠, meaning that llh₁ will be added to the sequence but an evaluation is not conducted. We move on to the next step and that we are now on llh₁. Again, we select the next low-level heuristic using the scores in the transition matrix to move from llh₁ and in this example, llh₄ is selected. The probability of selecting any of the two acceptance strategies for llh₄ is 1/2. Assume that the hyper-heuristic uses the roulette wheel selection strategy to select the acceptance strategy ⁠. This means that the sequence of heuristics is now constructed and it will be applied to the current solution. The constructed sequence is meaning that a sequence of heuristics of size 2 is generated. This sequence will be applied sequentially to the current solution, returning a new solution. Assume that the new solution is better than the current best obtained solution during the optimisation. In this case the relevant scores will be updated and increased by 1. The scores of moving from llh₂ to llh₁ and from llh₁ to llh₄ will be updated. Also the sequence-based acceptance strategy of llh₁ and the sequence-based acceptance strategy of llh₄ will be updated as illustrated in Figure 4. Note that if the new solution does not improve the quality of the best solution in hand, then the scores in the HMM matrices will not be updated. The next component of the hyper-heuristic is the move acceptance method. If, for example, we are using HC move acceptance, then the new solution will be accepted if its quality is better than the candidate solution; otherwise, the new solution will be rejected. We are now at llh₄, and we continue with the same strategy to construct and apply the next sequence of heuristics using the new scores and so on.

In this work we investigate the performance of three further variants of this method. They all adopt a similar selection strategy but differ in the reward mechanism.

• Sequence-Based Selection Hyper-Heuristic with Linear Update (SSHH-L): Corresponding score values are increased linearly by t as a reward mechanism as long as the best recorded solution in hand is improved; where t is the time elapsed in seconds.

• Sequence-Based Selection Hyper-Heuristic with Nonlinear Update (SSHH-N): Corresponding score values are increased nonlinearly by as a reward mechanism as long as the best recorded solution in hand is improved; where t is the time elapsed in seconds and c is a constant assigned arbitrarily to 30 in this work.

• Sequence-Based Selection Hyper-Heuristic with Delta Update (SSHH-D): Instead of a predefined rewarding mechanism, the difference in objective value between the newly generated solution and the best recorded solution after the application of the selected sequence of heuristics is used as a score value.

The experimentation is conducted on an i7-4770K CPU at 3.50 GHz with a memory of 16.00 GB. The Mann–Whitney–Wilcoxon test (Fagerland and Sandvik, 2009; Kruskal, 1957) is performed with a 95% confidence level in order to compare pairwise performance variations of two given algorithms statistically. The following notations are used: Given algorithm A versus algorithm B, (i) + (−) denotes that A (B) is better than B (A) and this performance variance is statistically significant, (ii) A B indicates that there is no statistical significant between A and B. In the first set of experiments, eight selected instances from the ITC 2011 set (one instance per each country) are used. The termination criterion is set to 200 seconds for the preliminary experiments.

Table 2 summarises the performance of the sequence-based selection (SS) method with the different move acceptance methods, where the SS prefix refers to the sequence-based selection approach and the suffixes refer to the various move acceptance methods. As expected, the results confirm that the move acceptance method does not influence the overall performance greatly. Based on the Mann–Whitney–Wilcoxon test with respect to the averages over ten runs on the selected instances, there are no statistically significant performance differences between the different methods on almost all instances. The only exception is on the South Africa-Woodlands2009 instance, at which simulated annealing seems to perform better than the others.

Based on the ranking strategy employed during the second round of ITC 2011, the sequence-based selection method, when combined with the record-to-record travel move acceptance method, seems to perform slightly better than the other hyper-heuristic methods, as shown in Figure 5. Hence, SS-RR is taken under consideration from this point onwards for further performance analysis, and we will refer to SS-RR as SSHH.

In this section, we describe a set of exhaustive experiments to determine the extent to which sequences of heuristics are useful in comparison to single heuristics and compare this with SSHH’s ability to discover these online. We implemented a simple hyper-heuristic to create and evaluate sequences of heuristics rather than applying single heuristics. Given n low-level heuristics {}, we form other sequences of heuristics of size 2 and size 3 and then invoke them successively. The total number of sequences of low-level heuristics is in the overall: {}, where denotes the sequence of applying LLH_i followed by LLH_j and followed by LLH_k.

We are able to present results for a sequence size of up to 3 where 3,615 permutations are investigated. There are 5,4240 permutations of four heuristics and an analysis of this space is too computationally complex to execute here. The proposed SSHH does not take the size of sequences as a parameter, but rather it learns the optimum size during the optimisation (in an online manner).

One thousand solutions are randomly generated and applied to each sequence to these generated solutions over eight ITC 2011 instances (a representative instance from each country). The utilisation rate for each sequence is equal but that does not mean that applying all these sequences would improve the quality of the input solutions. We therefore computed the utilisation rate considering only improving moves per sequence and for each instance. Table 3 provides the top five sequences that generate the highest utilisation rate of improving moves. From the table, we observe that sequences of size 3 are the dominants. This is perhaps to be expected as a greater movement in search space can be accomplished with the application of three well-selected heuristics, although poorer performance could also perhaps be expected if the heuristics are not well matched. It is clear from Table 3 that heuristic 5 is particularly important for generating new and better solutions in these problems, and the 3-fold combination of LLH5 appears top for 4/8 instances. However, it should also be noted that for the other half, LLH5 is best combined with other heuristics to deliver optimal performance. This study of heuristic sequence behaviour is important in determining heuristic performance in context as part of the overall search process, rather than as single application events.

Now that the potential for applying sequences of heuristics has been established, the performance of two hyper-heuristic methods is compared; one method has a fixed parametrised sequence size and the other is SSHH, which learns the optimum sequence lengths automatically during the search process. The selection method of the fixed parametrised hyper-heuristic (FPHH) method selects at each decision point the size of the sequence l randomly (limited to three in this work), and then selects randomly a sequence of size l and applies it to the candidate solution. The same record-to-record travel move acceptance method employed for SSHH is used in FPHH. Table 4 provides the ranking score of SSHH and FPHH for eight ITC 2011 selected instances over ten runs using the ITC 2011 ranking method. The table shows that SSHH performs the best in nine instances including two draws. This provides evidence that the size of the sequences of heuristics should not be fixed, but rather that the hyper-heuristic method should take the responsibility of detecting the optimum size of sequences during the optimisation.

The performance of the different variants of SSHH described in Section 4 are investigated over eight ITC 2011 instances (a representative instance from each country). Table 5 and Figure 6 summarise the performance of each variant based on ten runs for each selected instance using the ranking strategy employed in the second round of the ITC 2011 competition. The results show that SSHH-L with linear update performs overall better than SSHH, SSHH-N, and SSHH-D. SSHH-L generates the best score in four instances including one draw. SSHH-N obtains the best results in three instances including a draw. The ranking results put SSHH-D third with best score in two instances. SSHH performs worse than the other algorithms considering that the number of instances for which it produces the best ranking score is zero.

The best and the worst performing methods (SSHH-L and SSHH) are taken forward for the performance comparison to previously proposed approaches.

Figure 7 depicts the average heuristic utilisation rate using SSHH over ten runs of each low-level heuristic considering only invocations that generated improvement on the best-of-run solution and AS = 1 while solving two selected sample instances of Spain-School and South Africa-Woodlands2009. Figure 8 shows the same but considering both AS = 1 and AS = 2. It can be observed in Figure 7 that LLH0, LLH1, and LLH5 are the most successful heuristics, generating the highest utilisation rate in both instances. LLH2 seems to perform well in the Spain-School instance, and the same applies to LLH6 in the South Africa-Woodlands2009 instance. The remaining heuristics appear to provide poor performance. One may notice from Figure 7 that the most successful heuristics are event-oriented operators (LLH0-LLH8) rather than resource-oriented operators (LLH9-LLH14). Having said that and by examining Figure 8, the results show that resource-oriented operators are useful when combined and applied within a sequence of operations.

We repeat the same experiments for SSHH-L. Figures 9 and 10 provide the average utilisation rate when AS = 1 and AS = 1 and 2, respectively, while solving two selected sample instances of Spain-School and South Africa-Woodlands2009. Again and from Figure 9, LLH0, LLH1, and LLH5 are the most successful heuristics, generating the highest utilisation rate in both instances. We also observe from Figure 10 the involvement of resource-oriented operators showing that they are useful when combined and applied within a sequence of heuristics.

Figure 11 provides the average probabilities of the HMM matrices for each low-level heuristic over ten runs using SSHH while solving two selected sample problem instances of Spain-School and South Africa-Woodlands2009. By examining the transition figure for Spain-School instance, the following exploitative heuristics LLH0, LLH5, and LLH1 appear to be more successful than others in delivering best-of-run solutions as shown by the probability matrix in the South Africa-Woodlands2009 problem instance. In the latter, LLH9 is an exploration heuristic that needs to be combined with (preferably) LLH1.

Figure 12 shows the average probabilities of the HMM matrices for each low-level heuristic over ten runs using SSHH-L while solving the same two selected instances. In the Spain-School instance, the heuristic LLH1 seems to dominate the search process with the help of several exploration heuristics such as LLH2, LLH7, LLH11, and LLH13. Sets of likely sequences have been generated using a roulette wheel simulator (Algorithm 1) and the final HMM probability matrices as input for the Spain-School and South-Africa-Woodlands2009 problem instances. Table 6 summarises the results. Although single heuristics are frequently used, the approach clearly identifies sequences of size 2 and 3 as useful to the search. In the South Africa-Woodlands2009 instance, the sequences LLH14-LLH5 and LLH9-LLH0 are identified amongst the top ten generated sequences. An interesting finding is that the exploration heuristics LLH9 and LLH14 are resource-oriented operators whilst LLH0 and LLH5 are event-oriented operators. In the Spain-School instance, the sequence of size 3, LLH13-LLH9-LLH0, is also identified being generated 153 times. It should be noted here that this represents the set of sequences generated from the final learned matrix for each problem and the approach may also have identified alternative sequences during the search process.

Although the ITC 2011 set of instances is deprecated, we would like to obtain an approximate idea of the relative performance of SSHH and SSHH-L with respect to ITC 2011 solvers. For the comparisons with ITC 2011 solvers, a trial terminates after the equivalent to = 1,000 seconds is reached as the competition requires. A benchmarking software tool provided at the competition website is used to report the equivalent time value in the used machine. It reported that our machine should take 492 seconds per run. Using the results of the second round of ITC 2011 solvers provided at the competition website, Figure 13 summarises the performance comparison of SSHH and SSHH-L to the ITC 2011 methods (GOAL, HySST, Lectio, and HFT) over 18 ITC 2011 instances, each with ten runs. The results show that the SSHH-L method is the winner with a score of 2.22 in the overall, with SSHH slightly behind, indicating that the sequence selection–based approach produces best-in-class results on this problem set.

Table 7 summarises the results on XHSTT-2014 benchmark over ten runs. The solvers (SSHH and SSHH-L) terminate as long as there is no improvement to the best solution in hand for 200 seconds. At the time of submission, our solvers managed to deliver 9 best-known solutions, and match the best-known solutions in 4 instances. The performance is superior on most of the large instances including those from Australia, Denmark, the Netherlands, and the USA. They also perform well on the problem instances from Greece and Kosova. The performance on the instances from Brazil, Spain, Finland, England, and Italy is average but still our solvers can deliver near best-known solutions. However, they cannot generate the best or near the best-known solutions on the problem instances from South Africa.

Considering the averages performance, SSHH-L is the winner in 18 instances including 3 ties; while SSHH wins in 10 instances including 3 ties. Overall, the linear nature of SSHH-L and its ability to weight improvements later in the search as more important appears to preserve diversity and improve on the standard SSHH implementation.

The goal in hyper-heuristic research is to raise the level of generality by offering search methodologies that are easier to use and cheaper to implement and maintain than knowledge-intensive methods and yet deliver better average-case performance across a range of problems.

A sequence-based selection hyper-heuristic framework is introduced in this study whose selection component aims to discover sequences of heuristics.

In this article it has been shown:

• That the selection strategy of a hyper-heuristic is more important than the move acceptance method for this range of problems.

• That sequences of heuristics are able to deliver improved performance over single applications of a heuristic and that some problem instances benefit from the use of multiple heuristic types working together in sequence.

• That sequence lengths are better to be optimised on a per instance basis rather than randomly chosen.

• That an approach that prioritises heuristics that are able to generate better solutions towards the end of the search performs better than an approach that does not. This stands to reason because, as the search progresses, it becomes increasingly difficult to find solutions that achieve better performance than the current solution and so those heuristics capable of discovering these later in the search should be rewarded accordingly. This method also provides a mechanism for the hyper-heuristic to modify its learned behaviour later in the search, something that is possible, but less prevalent in the standard SSHH method.

In conclusion, the empirical results indicate that the proposed method is powerful and an effective general search methodology performing better than the current state-of-the-art methods in solving high school timetabling problems.

Future work will focus on the automated adaptation of the single parameter of the method within the move acceptance component.

This work was supported by EPSRC grant EP/K000519/1. The authors would like to thank David Walker for developing the simulator presented in the results section.