A Data Stream Ensemble Assisted Multifactorial Evolutionary Algorithm for Offline Data-Driven Dynamic Optimization

A common assumption underlying many optimization problems is that analytical objective and constraint functions exist. However, this assumption may not hold in many real-life situations, such as protein or molecule design problems (Brookes et al., 2019; Gaulton et al., 2012), and robot morphology design (Liao et al., 2019), where the objective function to be optimized cannot be described analytically (Jin and Sendhoff, 2009). In some cases, the objective or constraint values can be calculated based on only a certain amount of data collected, and such problems are known as data-driven optimization problems (Jin, 2016; Jin et al., 2019). Existing methods of solving data-driven optimization problems typically train a surrogate model from the collected data first. Then they use the surrogate model to replace real objectives or constraints to search for optimal solutions. During optimization processes, in cases where no candidate solutions can be verified by real objective or constraint functions, the problems are called offline data-driven optimization problems, which is different from online data-driven optimization problems (Jin et al., 2019; Wang et al., 2016). Evolutionary algorithms (EAs) are popular tools for data-driven optimization algorithms (Wang et al., 2016; Chugh et al., 2019) and are referred to as data-driven EAs. To solve online optimization problems, data-driven EAs generally replace expensive real fitness functions with less expensive/low-cost surrogates to save computational time or resources. In contrast, offline data-driven EAs can solely perform on surrogates since real fitness function evaluation is unavailable in offline data-driven optimization problems.

Even though many data-driven EAs have been developed (Jin et al., 2021), most of them focus on problems in static environments with the assumption of fixed real objective functions. However, complex real-world processes often work in dynamic scenarios where parameters in real objective functions or constraints evolve over time (Jin and Branke, 2005; Mavrovouniotis et al., 2017). For example, in operational indices optimization of the beneficiation process, raw ore type or equipment capacity may change during processing to meet market requirements (Ding et al., 2012). In these situations, the distributions of collected data change with time. We denote the offline data-driven optimization problems in such scenarios as offline data-driven optimization in dynamic environments (DynODD). Addressing DynODD requires adapting surrogate models to and tracking optimums of changing environments. Most recently, a data-driven EA for dealing with online data-driven optimization problems in dynamic environments, SAEF, has been proposed in Luo et al. (2018). In SAEF, surrogate models are rebuilt at each new environment with the newly collected data; then a memory scheme that reuses excellent solutions of the past environments is applied to initialize the population to achieve fast convergence. Generally speaking, surrogate models trained using data only of the current environment are not accurate enough in the case of small data. Meanwhile, an EA for DynODD must be able to quickly and accurately track the moving optimum. Thus, new optimization strategies for adapting to changes need to be designed.

As reported in Krawczyk et al. (2017) and Yazdani et al. (2021), a new environment is usually related to its previous environments in dynamic environments. Accordingly, the knowledge obtained from past environments may be helpful to the new environment. Therefore, techniques for extracting useful knowledge from historical environments have been widely investigated in both learning and optimization areas. For example, in the concept drift learning problems (Krawczyk et al., 2017; Gomes et al., 2017; Lu et al., 2018), where the statistical properties of the generated data change over time, methods for concept drift learning problems reuse data or the learned models of the historical environments to generate a high-quality model for the new environment (Alippi and Roveri, 2008; Street and Kim, 2001). In dynamic optimization problems (DOPs), the proposed evolutionary dynamic algorithms use solutions in the past environments to produce high-quality solutions for the new environment via memory scheme (Deb et al., 2007; Liu et al., 2010), prediction model (Cao et al., 2019; Muruganantham et al., 2015), or transfer learning method (Liu et al., 2019; Jiang et al., 2020).

Inspired by these learning and optimization approaches in dynamic environments, this study aims to address the challenges mentioned above in DynODD by taking full advantage of historical knowledge. To this end, we first adopt concept drift learning approaches to construct surrogate models for the data collected from dynamic environments. Here, we employ an effective data stream ensemble (DSE) (Gomes et al., 2017) approach to tracking the concept drift data. DSE is a combination of a set of base learners, where each base learner approximates an objective function in a selected number of past environments. After that, we search for optimal solutions for the real fitness functions by simultaneously optimizing the selected past base learners and the ensemble surrogate model in a multitask environment via a multitask EA, also known as multifactorial evolutionary algorithms (MFEA). MFEA enables knowledge sharing among the multiple optimization tasks, thereby improving the efficiency and/or effectiveness of solving each task (Gupta, Ong, and Feng, 2016). In this work, we optimize both the DSE surrogate model and its base learners, aiming at transferring useful knowledge acquired by the base learners to the DSE-assistant optimization. By doing so, we attempt to transfer knowledge of past environments to improve the optimal solution of the current environment searching, thereby accelerating optimum tracking speed. Unlike the classical MFEA (Gupta, Ong, and Feng, 2016), this work keeps more individuals for the ensemble surrogate model than its base learners, since we focus on optimizing the ensemble surrogate model. The proposed offline data-driven optimization algorithm is termed DSE-assisted MFEA, DSE_MFS for short. Contributions of this work can be summarized as follows:

1. We propose a knowledge-transfer-based method for offline data-driven evolutionary optimization in dynamic environments, where knowledge transfer is carried out in surrogate model construction and surrogate-assisted optimization procedures;

2. We employ data stream ensemble learning to build DSE surrogate models in each environment of DynODD. The data stream ensemble learning combines base learners trained by historical data, thus being able to leverage the knowledge of historical environments;

3. We simultaneously optimize the DSE surrogate and its base learners via MFEA to extract knowledge from historical environments to speed up the DSE surrogate-assisted optimization. In MFEA, the DSE surrogate is assigned with more individuals than its base learners to improve searchability.

The rest of the paper is organized as follows. The definition of DynODD, backgrounds of learning in dynamic environments, MFEA, and data-driven EAs are introduced in Section 2. The proposed algorithm is described in detail in Section 3. Experimental studies, simulation results, and discussions are presented in Section 4. Finally, the conclusions and possible future research on DynODD are outlined in Section 5.

Data-driven optimization problems refer to problems where analytical optimization functions of objectives and constraints are unavailable. Thus, optimization approaches for solving these problems often rely on collected data (Jin et al., 2021). Surrogate-assist optimization algorithms are dominating methods for data-driven optimization problems. Their main idea is building a surrogate model based on the data and using this surrogate model as the objective function to evaluate solutions. Various strategies for building and managing surrogate models and searching strategies have been proposed in the past years (Jin and Sendhoff, 2009). In cases where simulation platforms are available, the optimization can be directly driven by the simulation platforms. Usually, multiple simulators with different evaluation accuracy and computation costs can be obtained for a certain problem. To balance evaluation accuracy and computational costs, multifidelity optimization methods (Balabanov and Venter, 2004; Zimmer et al., 2021; Branke et al., 2016) and fidelity-adaptive strategies (Conn and Le Digabel, 2013) have been investigated in the past decades.

Offline data-driven EAs mainly focus on surrogate modeling, surrogate model-assisted optimization, and model management by efficiently and effectively using collected data because no new data can be actively sampled (Jin et al., 2019, 2021). According to characters of collected data, offline data-driven EAs vary a lot. For example, Wang et al. (2016) proposed a multifidelity surrogate model management to deal with problems involving a large amount of data, which will result in prohibitively large recourse or computational cost in fitness evaluation. The algorithm reduces computational costs while preserving an acceptable optimal solution by switching different fidelity models. Chugh et al. (2017) suggested a local regression smoothing technique to preprocess the data before surrogate model formulation for handling small offline data-driven problems containing noise and outliers. They employed KRVEA (Chugh et al., 2018) to perform an optimization search based on the constructed models. To solve problems with small data, an NSGAII_GP algorithm that consists of low-order PR and GP models is developed by Guo et al. (2016). The low-order PR model serves as a real fitness function to generate synthetic data for alleviating limited data. In contrast, the GP model performs an optimization search based on offline and synthetic data. Wang et al. (2019) presented a selective ensemble surrogate algorithm for problems with limited data. Yang et al. (2019) developed a coarse-fine surrogate model, where the coarse surrogate aims to guide the algorithm to find a promising subregion quickly, and the fine model focuses on leveraging good solutions according to the knowledge transferred from the coarse surrogate. Meanwhile, they suggested a reference-based average technique for generating final solutions. Li, Zhan, Wang, Jin, et al. (2020) presented boosting data-driven EAs. They first iteratively built new models and combined them as the surrogate model, then proposed a localized data generation strategy to generate synthetic data to increase data and its quantity. Huang et al. (2021) introduced semisupervised learning to the optimization process and adopted a tri-training strategy to manage surrogate models. Li, Zhan, Wang, and Zhang (2020) proposed a perturbation-based ensemble surrogate that first generates diverse surrogates based on perturbed data and then selects some of the built surrogates to form an ensemble surrogate model.

In addition to the offline data-driven optimization methods that use supervised learning to construct surrogate models, other strategies concerning off-manifold, invalid, and low-scoring out-of-distribution (OOD) data have been proposed to achieve a robust surrogate model. For example, Model Inversion Networks (MIN) (Kumar and Levine, 2020) learn an inverse function instead of training a surrogate model, mapping objective values to decision variables, in order to avoid searching OOD solutions. Similar to MIN, a density estimator is employed in Brookes et al. (2019) to restrict the searched solution to be in distribution. In Trabucco et al. (2021), the sampled OOD data are applied to train a conservative surrogate model in order to prevent the surrogate model from overestimating these OOD data.

The data in many applications are naturally generated in a stream fashion, where data is continuously incoming, known as data streams. In the case of nonstationary data-generating processes, the statistical distributions of data streams will change. This phenomenon is called concept drift. An important factor of learning under concept drift is designing adaptive mechanisms to adapt to the newly available data. One kind of adaptive mechanism retains a single model in each new environment with the newest data. For example, Alippi and Roveri (2008) adopted a windowing strategy to preserve the most recent data for retraining the model, where the change ratio decides the window length. Then, adaptive length windowing techniques are proposed, whose length is estimated according to the rate of change (Bifet and Gavalda, 2007) or the ICI-based refinement procedure (Alippi et al., 2013). An alternative mechanism is the weighting mechanism, which uses all available samples to retain the model but weights each sample. The weights of the samples linearly decrease with time (Koychev, 2000) or are calculated based on decay functions (Cohen and Strauss, 2003). Different from the above two strategies, Aggarwal (2006) applied a reservoir sampling to select a subset of data to retrain the new model.

Recently, ensemble learning has attracted extensive attention to handling concepts. The developed data stream ensemble (DSE) methods differ in selecting base classifiers and combining them (Krawczyk et al., 2017; Gomes et al., 2017; Lu et al., 2018). For example, Streaming Ensemble Algorithm (SEA) (Street and Kim, 2001) creates a classifier with the newest data chunk and removes the poorest historical classifier if the maximum number of archived historical classifiers is reached. The performance of historical classifiers is measured according to their predicted mean square errors on the current data chunk. Then, SEA combines all classifiers via a weighted sum scheme, which is calculated based on the classifier's performance. Accuracy Weighted Ensemble (AWE) differs from SEA in historical classifier selection. It maintains only classifiers whose prediction accuracies are higher than that of a randomly selected one. Alternatively, Accuracy Updated Ensemble (AUE2) (Brzezinski and Stefanowski, 2013) not only selects historical classifiers for combination but also incrementally updates these classifiers with the new data chunk. Sun et al. (2018) selects a pool of individual classifiers with diversified qualities in order to cover different changes. Similarly, diverse instance-weighting ensemble (DiwE) (Liu et al., 2020) maintains diversified classifiers according to their agreement on the probability of a regional distribution change.

In solving dynamic optimization problems (DOPs), EAs usually use information in historical environments to adapt to the new environment instead of restarting the optimization, which can hopefully enhance the search efficiency since problems before and after a change are somehow related (Nakano et al., 2015; Nasiri and Meybodi, 2016). Many approaches have been proposed for adapting to changes. For example, the work in Deb et al. (2007), Cobb (1990), and Liu et al. (2010) introduces a hypermutation, which increases the mutation rate at the beginning of a new environment, and then gradually decreases the mutation rate to enable the population to converge to the optimal location. Their basic idea of diversity enhancement is introducing diversity to the converged population to help it jump out of the current optimum, thereby facilitating new optimum searching. Reusing the optimal solutions in previous environments would accelerate the convergence speed in the situation of recurrent or periodical changes. For instance, the most diversified solutions (Yu and Suganthan, 2009) or the best solutions (Daneshyari and Yen, 2011) in a particular historical moment are injected into new populations. In addition, prediction-based approaches are often adopted when the changes exhibit a regular pattern. It first recognizes the moving pattern via knowledge in previous environments, then initializes the population that would be close to the optimum of the new environment using this pattern. For example, the work in Cao et al. (2019), Muruganantham et al. (2015), and Hatzakis and Wallace (2006) estimates the moving path via Support Vector Regression, Kalman Filtering, Autoregressive model. In recent years, transfer learning techniques have been employed to acquire knowledge in the previous environments and transfer it to the new environment to enhance search efficiency (Liu et al., 2019; Jiang et al., 2020).

Multitask evolutionary optimization algorithm (MFEA) (Gupta, Ong, and Feng, 2016), is a paradigm that concurrently solves multiple optimization problems (tasks) using one population. It is expected to enhance problem solving via knowledge transfer among tasks. Similar to traditional EAs, procedures of MFEA include population initialization and evaluation, reproduction, offspring evaluation, and environment selection. Major different procedures between MFEA and traditional EAs are listed as follows (Gupta, Ong, and Feng, 2016):

• Each individual is assigned a skill factor $τ$ except gene in the population initialization, representing the aligned task.

• In evaluation processes, individuals are evaluated only by their aligned tasks according to the skill factor.

• The reproduction in MFEA involves an assortative mating procedure and a vertical cultural transmission procedure. In particular, assortative mating allows individuals from different tasks to mate with each other to create offspring. Thereafter, the vertical cultural transmission assigns skill factors to the generated offspring according to their parents.

• Environment selection is performed for each task independently to select individuals of the next generation.

Many recent multitask evolutionary optimization algorithms have been proposed to enhance positive knowledge transfer and reduce negative knowledge transfer among tasks. For example, a data-driven multitasking approach is proposed (Bali et al., 2019) to determine how much knowledge can be transferred between tasks dynamically. Li et al. (2021) developed a meta-knowledge transfer strategy to leverage knowledge from heterogeneous multisource data to the target task. Wu et al. (2022) proposed an approach for knowledge transfer among heterogeneous tasks. On the one hand, they mapped the global best individual of the source task from its original search space to the search space of the target task via an optimization process to handle the difference in task dimension. On the other hand, they proposed an orthogonal transfer to find the best combination of different dimensions across the two heterogeneous tasks. We employ the simplest MFEA approaches (Gupta, Ong, and Feng, 2016) in this work, while the above advanced knowledge transfer methods are valued to be investigated in our DSE surrogate-assisted optimization in future work.

In this section, we first present the main framework of the proposed DSE_MFS. Then, we detail its two components, DSE surrogate construction and DSE surrogate-assisted MFEA. Finally, we analyze the computational complexity analysis of the proposed DSE_MFS.

The DSE surrogate is constructed using the AUE2 algorithm. AUE2 was developed to solve classification problems in which the base learner is a Hoeffding tree. In this work, we replace the Hoeffding tree with a radial-basis-function (RBF) network for regression tasks. The pseudocode of the revised AUE2 is presented in Algorithm 1.

If the number of the existing base learners, $K$⁠, is less than the predefined maximum number $Kmax$⁠, the new member $St$ is simply added to the ensemble. Otherwise, a base learner with the largest RMSE in the $K$ will be replaced by $St$⁠. Similar to Brzezinski and Stefanowski (2013), $Kmax$ is set to 10 in this work.

Apart from adjusting the weight of each base learner, AUE2 also updates all base learner except for $St$⁠, using $Bt$ (lines 18–21 in Algorithm 1). This is achieved by updating each base learner using a combination of its own training data and the most $Bt$⁠. For example, base learner $Sk$ is updated using a combination of $Bk$ and $Bt$⁠. Meanwhile, the training data of $Sk$ is replaced with the union of $Bk$ and $Bt$⁠.

Note that the surrogate for assisting the optimization of the $t$-th environment is $DSEt$ consisting of $Sk,k=1,2,...,K$ and $St$⁠. When $t=1$⁠, however, the $DSE1$ is the same as $S1$⁠.

The proposed DSE_MFS solves $K+1$ problems including the DSE surrogate and its base learners simultaneously using MFEA (Gupta, Ong, and Feng, 2016). By doing this, the knowledge about the fitness landscapes of the previous problems acquired by the ensemble base learners can be transferred to accelerate solution searching of the current problem. Different from existing MFEAs (Gupta, Ong, and Feng, 2016; Ding et al., 2017; Bali et al., 2019; Gupta, Ong, Feng, and Tan, 2016) that treat all tasks equally important, DSE_MFS aims to find the optimum of the current environment. Therefore, we assign a larger number of individuals for the current optimization problem than those of the previous problems. Specifically, for a given population size $NP$⁠, $NP2$ individuals are assigned to solving the DSE-assisted optimization of the current problem. By contrast, $NP2K$ individuals are related to each base learner, where $K$ is the number of base learners. All individuals in the population are randomly initialized and assigned to a task via a skill factor $τ$⁠. An individual $pi$ is assigned to the $k$-th base learner if its skill factor $τi=k,k∈{1,2,...,K}$⁠; otherwise to the DSE surrogate if $τi=K+1$⁠.

Like other MFEAs, knowledge transfer from previous problems to current optimization problems is realized during reproduction, including assortative mating and vertical cultural transmission. Details of the offspring creation process in the proposed DSE_MFS are presented in Algorithm 2. For a pair of parents $pa,pb$⁠, whose skill factors are $τa,τb$⁠, their offspring individuals are generated with crossover and mutation if the two parents come from the same task, and they are assigned to the same task of their parents (lines 3–5). If the parents are from different tasks, a randomly generated number is smaller than the predefined random mating probability (⁠$rmp$⁠). The knowledge between the two tasks can also be shared in the generated offspring through crossover and mutation. Notably, the probability of assigning an offspring to a task is different from MFEA, in which it is fixed. In this work, DSE_MFS adaptively calculates the probability in accordance with each task to maintain their associated individual number constant. For example, if the number of individuals for tasks $τa$ and $τb$ are $Na$ and $Nb$⁠, respectively, then the probability of assigning an offspring to $τa(τca$⁠) is $NaNa+Nb$ and that to $τb(τcb$⁠) is $NbNa+Nb$⁠. If a randomly generated number is smaller than $rmp$⁠, then the offspring is created by mutating one of the parents; this offspring is assigned to the same task as its parent accordingly, as shown in lines 18–23. Once the offspring are generated and evaluated by the corresponding tasks, environment selection is performed for each task to select parents for the next generation.

The DSE surrogate-assisted MFEA simultaneously optimizes multiple objectives, that is, the DSE surrogate model and its base learners, making the proposed DSE_MFS similar to dynamic multiobjective optimization algorithms. However, they are very different, as at each environment, DSE_MFS aims to find only one optimal solution corresponding to the DSE surrogate model with the assistance of optimizing the other objectives. In contrast, dynamic multiobjective optimization algorithms aim to find a set of Pareto-set that trade off across multiple objectives. Although the proposed algorithm in this work focuses on single-objective DynODD, it can be extended to multiobjective DynODD in a straightforward way by building and managing surrogate models for each objective and replacing the multitask search algorithm with a multi-objective multitask search algorithm, such as Mo-MFEA (Gupta, Ong, Feng, and Tan, 2016) or ATO-MFEA (Yang et al., 2017). As analyzed in Gupta et al. (2018), an exchange of genetic materials between tasks in the multitask environment enables each task to automatically leverage useful knowledge from other tasks while preserving its own best genes. Meanwhile, each base learner exhibits a specific search behavior because of the time-varying objective functions each base learner aims to approximate. Therefore, the base learners can continuously transfer knowledge of the previous problems to the current environment to make an efficient search.

Suppose there are $T$ environments in the data generation process, and $N$ data are generated at each environment. The number of center points in RBF is $C$⁠, the number of generations is $G$⁠, and the population size of MFEA is $Q$⁠. Then, the time complexity of the DSE_MFS, including surrogate model training, the best base learners evaluation and selection, and surrogate model prediction, are analyzed as follows.

As for base learner selection, DSE_MFS needs to evaluate all base learners in the archive on the newly generated data, so the time complexity is $O(NC(Kmax-1)Kmax)$⁠. Meanwhile, DSE_MFS needs to sort all archived base learners, so the time complexity is $O(Kmaxlog(Kmax)(T-Kmax))$⁠.

To investigate the performance of the proposed DSE_MFS algorithm in solving DynODD, we compare it with three state-of-the-art offline data-driven EAs for stationary optimization, namely, DDEA-SE (Wang et al., 2019), TT-DDEA (Huang et al., 2021), and BDDEA-LDG (Li, Zhan, Wang, Jin et al., 2020), and one online data-driven EA for dynamic optimization, SAEF (Luo et al., 2018). Among the compared algorithms, DDEA-SE is an ensemble surrogate-assisted offline data-driven EA that selects the best base learners from a large pool of base learners to serve as the surrogate. TT-DDEA introduces a tri-training to update surrogate models with generated candidate solutions. BDDEA-LDG applies a boosting strategy to incrementally build surrogates and a localized data generation to generate synthetic data. SAEF rebuilds surrogate models at each environment and uses a memory-based strategy to track moving optimum. In SAEF, we apply a multipopulation particle swarm optimization-based method as an optimizer.

To implement offline data-driven EAs in DynODD, the data chunk generated from each environment is incrementally added to the training set. Then, the updated dataset is used to retrain each component. Since SAEF (Luo et al., 2018) is designed for online data-driven optimization, it needs real function evaluations when updating the surrogates. To adapt SAEF to offline DynODD, we remove the surrogate update part during the optimization. Among the seven algorithms in SAEF, we select the 1S_GP that uses GPs as surrogate models for comparison due to its effectiveness on ten-dimensional problems (Luo et al., 2018).

The CEC 2009 benchmark problems for dynamic optimization, which are generated from a GDBG system (Li et al., 2008), are used to evaluate the performance of the compared algorithms. This test suite contains six problems, F1–F6, where F1 is a maximization problem and F2–F6 are minimization problems. The characters of test instances are presented as follows:

• F1: Rotation peak function with 12 peaks.

• F2: Composition of ten sphere functions.

• F3: Composition of ten Rastrigin functions.

• F4: Composition of ten Griewank functions.

• F5: Composition of ten Ackley functions.

• F6: Hybrid composition function consists of sphere, Rastrigin, Weierstrass, Griewank, and Ackley.

According to the change type of control parameters in the function, which determines the position of the best solution, GDBG designs six environmental change types for each test problem, which are listed as follows:

• C1: Small step change, where $φ$ changes mildly between two consecutive environments.

• C2: Large step change, where $φ$ has a significant change between two consecutive environments.

• C3: Random change, where $φ$ changes randomly.

• C4: Chaotic change, where the control parameters between two consecutives times related non-linearly.

• C5: Recurrent change, where the control parameters' period changed.

• C6: Recurrent with noise, where the control parameter change is caused by noise.

Parameters of the benchmark problems and the algorithms under comparison are as follows:

• The total number of environments $T$ in each independent run is set to $T=60$⁠. The maximum generation of each environment, that is, the change frequency $FEs$⁠, is set to 20.

• The distribution indexes of simulated binary crossover and polynomial mutation in EA are set to 20. The crossover probability and mutation probability are set to $pc=1.0$ and $pm=1/D$⁠, where $D$ is the number of decision variables. The randomly mating probability $rmp$ is set to $rmp=0.3$⁠, similar to MFEA.

RBF in the proposed DSE_MFS is constructed using the toolbox in Jekabsons (2009). Solutions obtained by the compared algorithms at each environment are re-evaluated using the real fitness functions before performance metrics are calculated. All experiments are executed on Matlab R2018a, Intel Xeon with 3.5 GHz CPU, Microsoft Windows 10 64-bit operating system.

In this section, we compare the performance of DDEA-SE, TT-DDEA, BDDEA-LDG, SAEF, and DSE_MFS in solving DynODD (Li, Zhan, Wang, Jin et al., 2020). In this experiment, each data chunk consists of $5D$ samples generated using Latin hypercube sampling (LHS) (Stein, 1987). These data are used to train or update DSE surrogate models. The statistical $E_BBC$ and $STD$ results of the five compared algorithms on ten-dimensional benchmark problems over 20 independent runs are presented in Table 1. Wilcoxon's rank sum test at a 0.05 significance level is performed to denote the significance of differences between the compared algorithms and DSE_MFS. In the table, the best metric values are highlighted. Symbols “$+$⁠,” “$-$⁠,” and “$=$” indicate the corresponding algorithm performs significantly better than, worse than, and comparable to the proposed DSE_MFS, respectively.

The table shows that DSE_MFS achieves minimum $E_BBC$ and $STD$ metric values on nearly all test problems among the five compared data-driven algorithms and performs a little worse than the other four compared algorithms on five functions. The results demonstrate a clear superior performance of DSE_MFS to that of the other four compared algorithms in dealing with DynODD. DDEA-SE, TT-DDEA, and BDDEA-LDG achieve higher $E_BBC$ for offline data-driven algorithms than DSE_MFS. The underlying reason may be that without an adaptation strategy, they cannot track the dynamic environments in DynODD. With closer observation, BDDEA-LDG performs the best among these three offline data-driven algorithms. This is because BDDEA-LDG applies a boosting strategy in surrogate model management, which can alleviate concept drift in building surrogate models. On the contrary, SAEF performs much worse than DSE_MFS. This may be attributed to the fact that SAEF uses only the data chunk of the current environment to build surrogates, which may lead to much larger approximation errors on problems with a small amount of data chunk. The results of SAEF suggest that it is important to reuse historical data to alleviate data shortage for small data-driven optimization problems.

We list the runtime of each compared algorithm on each test instance averaged over 20 independent runs in Table 1. These results show that SAEF and DSE_MFS spend much less time than the other three offline data-driven EAs. This is reasonable as SAEF and DSE_MFS update models only when the environment changes and do not consider model management strategies during each environment, thus saving much computational time.

As introduced in Section 3, DSE_MFS contains two main components, DSE surrogate model construction and ensemble surrogate-assisted MFEA. In the following, we perform additional experiments on ten-dimensional F1 and F6 to examine the role of each component by comparing three variants of DSE_MFS.

• SA_SOEA: SA_SOEA is a variant of DSE_MFS that replaces the multitask evolutionary optimizer with a traditional evolutionary algorithm and constructs the surrogate model using all data collected from the current and historical environments.

• DSE_SOEA: DSE_SOEA is a variant of DSE_MFS that replaces the multitask evolutionary optimizer with a traditional evolutionary algorithm but builds surrogate models using DSE.

• DSE_MPSO: DSE_MPSO is a variant of DSE_MFS that replaces the multitask evolutionary optimizer with a multipopulation particle swarm optimization algorithm proposed by Blackwell et al. (2008).

The statistical results of $E_BBC$ and $STD$ of the three variants on F1 and F6 are summarized in Table 2. Similarly, symbols “$+$⁠,” “$-$⁠,” and “$=$” indicate the corresponding variant is significantly better than, worse than, or comparable to DSE_MFS, respectively. The compared results between SA_SOEA, DSE_SOEA, DSE_MPSO, and DSE_MFS are listed in the last line of Table 2. In the table, the result $l/t/w$ denotes the corresponding variant in the current column loses $l$ times, ties $t$ times, and wins $w$ times compared with the corresponding variant in the next column. DSE_MFS achieves the best performance compared with its three variants. It should be emphasized that the three variants perform worse than or comparable to DSE_MFS on almost all the test problems. From the results in Table 2, we can make the following observations:

In this study, a data stream ensemble (DSE) assisted multifactorial evolutionary algorithm (MFEA) is proposed to solve dynamic offline data-driven optimization problems (DynODD). The DSE-assisted MFEA is able to leverage the knowledge of the problems in the previous environments to track the moving optimum quickly even if the amount of available data in each environment is small. The proposed algorithm, DSE_MFS, is compared with four state-of-the-art data-driven EAs on six dynamic optimization benchmarks to validate its performance. The empirical results demonstrate that DSE_MFS has achieved the best overall performance against the compared algorithms.

Although the performance of the proposed algorithm is encouraging, many challenges remain to be addressed. For example, this work reuses the knowledge acquired in previous environments in surrogate modeling and multitask optimization. Furthermore, investigating the use of surrogate models to generate a more informed initial population for preventing cold start is also interesting. Moreover, the dimension of the search space considered in this work is relatively low, and it will become much more challenging for high-dimensional problems. Finally, offline data-driven multi- and many-objective optimization in dynamic environments will also be important for future research.