## Abstract

Multiobjective evolutionary algorithms (MOEAs) have progressed significantly in recent decades, but most of them are designed to solve unconstrained multiobjective optimization problems. In fact, many real-world multiobjective problems contain a number of constraints. To promote research on constrained multiobjective optimization, we first propose a problem classification scheme with three primary types of difficulty, which reflect various types of challenges presented by real-world optimization problems, in order to characterize the constraint functions in constrained multiobjective optimization problems (CMOPs). These are feasibility-hardness, convergence-hardness, and diversity-hardness. We then develop a general toolkit to construct difficulty adjustable and scalable CMOPs (DAS-CMOPs, or DAS-CMaOPs when the number of objectives is greater than three) with three types of parameterized constraint functions developed to capture the three proposed types of difficulty. In fact, the combination of the three primary constraint functions with different parameters allows the construction of a large variety of CMOPs, with difficulty that can be defined by a triplet, with each of its parameters specifying the level of one of the types of primary difficulty. Furthermore, the number of objectives in this toolkit can be scaled beyond three. Based on this toolkit, we suggest nine difficulty adjustable and scalable CMOPs and nine CMaOPs, to be called DAS-CMOP1-9 and DAS-CMaOP1-9, respectively. To evaluate the proposed test problems, two popular CMOEAs—MOEA/D-CDP (MOEA/D with constraint dominance principle) and NSGA-II-CDP (NSGA-II with constraint dominance principle) and two popular constrained many-objective evolutionary algorithms (CMaOEAs)—C-MOEA/DD and C-NSGA-III—are used to compare performance on DAS-CMOP1-9 and DAS-CMaOP1-9 with a variety of difficulty triplets, respectively. The experimental results reveal that mechanisms in MOEA/D-CDP may be more effective in solving convergence-hard DAS-CMOPs, while mechanisms of NSGA-II-CDP may be more effective in solving DAS-CMOPs with simultaneous diversity-, feasibility-, and convergence-hardness. Mechanisms in C-NSGA-III may be more effective in solving feasibility-hard CMaOPs, while mechanisms of C-MOEA/DD may be more effective in solving CMaOPs with convergence-hardness. In addition, none of them can solve these problems efficiently, which stimulates us to continue to develop new CMOEAs and CMaOEAs to solve the suggested DAS-CMOPs and DAS-CMaOPs.

## 1  Introduction

Practical optimization problems usually involve simultaneous optimization of multiple conflicting objectives with many constraints. Without loss of generality, constrained multiobjective optimization problems (CMOPs) can be defined as follows:
$minimizeF(x)=(f1(x),…,fm(x))Tsubjecttogi(x)≥0,i=1,…,qhj(x)=0,j=1,…,px∈Rn,$
(1)
where $F(x)=(f1(x),f2(x),…,fm(x))T∈Rm$ is an $m$-dimensional objective vector, $gi(x)≥0$ defines the $i$-th of $q$ inequality constraints, and $hj(x)=0$ defines the $j$-th of $p$ equality constraints. If $m$ is greater than three, we usually call it a constrained many-objective optimization problem (CMaOP).

A solution $x$ is said to be feasible if it meets $gi(x)≥0,i=1,…,q$ and $hj(x)=0,j=1,…,p$ at the same time. For two feasible solutions $x1$ and $x2$, solution $x1$ is said to dominate $x2$ if $fi(x1)≤fi(x2)$ for each $i∈{1,…,m}$ and $fj(x1) for at least one $j∈{1,…,m}$, denoted as $x1≼x2$. For a feasible solution $x*∈Rn$, if there is no other feasible solution $x∈Rn$ dominating $x*$, $x*$ is said to be a feasible Pareto-optimal solution. The set of all feasible Pareto-optimal solutions is called the Pareto set ($PS$). Mapping the $PS$ into the objective space defines a set of objective vectors, denoted as the Pareto front (PF), where $PF={F(x)∈Rm|x∈PS}$.

For CMOPs, more than one objective must be optimized simultaneously, subject to constraints. Generally speaking, CMOPs are much more difficult to solve than their unconstrained counterparts—unconstrained multiobjective optimization problems (MOPs). Constrained multiobjective evolutionary algorithms (CMOEAs) are particularly designed to solve CMOPs, with the capability of balancing the search between the feasible and infeasible regions in the search space (Runarsson and Yao, 2005). In fact, two basic issues must be considered carefully when designing a CMOEA. One is to manage the effort devoted to seeking feasible, rather than infeasible, solutions; the other is to balance the convergence and diversity of the CMOEA.

To address the first issue, constraint handling mechanisms must be carefully designed by researchers. The existing constraint-handling methods can be broadly classified into five different types: penalty functions, special representations and operators, repair algorithms, separation of constraints and objectives, and ensemble-of-constraint-handling-methods approaches, as delineated in Coello Coello (2002), Mezura-Montes and Coello (2011), and Coello (2018). The penalty function-based method is one of the most popular of such approaches. The overall constraint violation is added to each objective with a predefined penalty factor, which indicates a preference between the satisfaction of the constraints and the extremization of the objectives. Penalty function-based methods include use of a death penalty (Hoffmeister and Sprave, 1996), static penalty (Hoffmeister and Sprave, 1996), dynamic penalty (Joines and Houck, 1994), adaptive penalty (Coit et al., 1996; Ben Hadj-Alouane and Bean, 1997), co-evolutionary penalty (Huang et al., 2007), and self-adaptive penalty (Tessema and Yen, 2006; Woldesenbet et al., 2009). The methods using special representations and operators try to map chromosomes from the infeasible region into the feasible region. A more interesting type of approach within this group is the so-called ”Decoders” method, which is based on the idea of mapping the feasible region into an easier-to-sample space where an evolutionary algorithm can provide better performance (Koziel and Michalewicz, 1998). Representative examples of this type include homomorphous maps (HM) (Koziel and Michalewicz, 1999) and Riemann mapping (Kim, 1998). The third type, repair algorithms, try to transform an infeasible solution into a feasible one. A typical example of this type is Genocop III (genetic algorithm for numerical optimization of constrained problems) (Michalewicz and Nazhiyath, 1995). However, this type of approach is problem-dependent and is usually used in combinatorial optimization problems (Salcedo-Sanz, 2009). For a particular problem, a specific repair algorithm must be designed. In the methods using separation of constraints and objectives, the constraint functions and the objective functions are treated separately. Some examples of this type include coevolution (Paredis, 1994), constraint dominance principle (CDP) (Deb, 2000; Deb et al., 2001), stochastic ranking (Runarsson and Yao, 2000), $α$-constrained method (also called the $ɛ$-constrained method) (Takahama and Sakai, 2005; Laumanns et al., 2006; Takahama et al., 2005), use of multiobjective optimization concepts (Cai and Wang, 2006; Ray et al., 2009), and COMOGA (Surry and Radcliffe, 1997). The ensemble-of-constraint-handling-methods approaches usually adopt several constraint-handling methods to deal with constraints. Representative methods include the adaptive trade-off model (ATM) (Wang et al., 2008) and the ensemble of constraint handling methods (ECHM) (Mallipeddi and Suganthan, 2010; Qu and Suganthan, 2011).

To address the second issue, selection methods need to be designed to balance the performance in terms of convergence versus the diversity in an MOEA. At present, MOEAs can be generally classified into three categories based on selection strategies. They are Pareto-dominance-based (e.g., NSGA-II (Deb et al., 2002), PAES-II (Corne et al., 2001), and SPEA-II (Zitzler et al., 2001)), decomposition-based (e.g., MOEA/D (Zhang and Li, 2007), MOEA/D-DE (Li and Zhang, 2009), MOEA/D-M2M (Liu et al., 2014), and EAG-MOEA/D (Cai et al., 2015)) and indicator-based (e.g., IBEA (Zitzler and Künzli, 2004), R2-IBEA (Phan and Suzuki, 2013), SMS-EMOA (Beume et al., 2007), and HypE (Bader and Zitzler, 2011)). In the group of Pareto-dominance-based methods, such as NSGA-II (Deb et al., 2002), the first set of nondominated solutions is selected to improve the convergence performance, and a crowding distance measure in the last-ranked nondominated set is adopted to maintain the diversity performance. In decomposition-based methods, the convergence performance is maintained by minimizing the aggregation functions and the diversity performance is obtained by setting the weight vectors uniformly. In indicator-based methods, such as HypE (Bader and Zitzler, 2011), the convergence and diversity performance are both achieved using a hypervolume metric.

A CMOP includes objectives and constraints. A number of features have already been identified to define the difficulty of objectives, which include:

1. Geometry of a PF (linear, convex, concave, degenerate, disconnected, or a mixture of them)

2. Search space (biased, or unbiased)

3. Unimodal or multimodal objectives

4. Dimensionality of variable space and objective space

The first of these features is the geometry of the PF. The PF of an MOP can be linear, convex, concave, degenerate, disconnected, or a mixture of them. Representative benchmarking MOPs reflecting this type of difficulty include ZDT (Deb, 1999), F1-9 (Li and Zhang, 2009), and DTLZ (Deb et al., 2005). The second feature is the biased or unbiased nature of the search space, which means whether an evenly distributed sample of decision vectors in the search space maps to an evenly distributed set of objective vectors in the objective space (Huband et al., 2006). Representative benchmarking MOPs with biased search spaces include MOP1-7 (Liu et al., 2014) and IMB1-14 (Liu et al., 2017). The third feature is the modality of objectives. The objectives of an MOP can be either unimodal (for example, DTLZ1 (Deb et al., 2005)) or multimodal (for example, F8 (Li and Zhang, 2009)). Multimodal objectives have multiple locally optimal solutions, which increases the likelihood of an algorithm being trapped in a local optimum. High dimensionality of variable space and objective space are also critical features in defining the difficulty of objectives. LSMOP1-9 (Cheng et al., 2017) have high dimensionality in the variable space. DTLZ (Deb et al., 2005) and WFG (Huband et al., 2006) have high dimensionality in the objective space.

On the other hand, constraint functions in general greatly increase the difficulty of solving CMOPs. However, as far as we know, only relatively few test suites (CTP (Deb, 2001) and CF (Zhang et al., 2008)) have been designed for CMOPs.

The CTP test problems (Deb, 2001) allow adjusting of the difficulty of the constraint functions. They offer two types of difficulties: the difficulty near the PF and the difficulty in the entire search space. The test problem CTP1 gives difficulty near the PF, because the constraint functions of CTP1 make the search region near the Pareto front infeasible. Test problems CTP2-CTP8 provide difficulty for an optimizer throughout the search space.

The CF test problems (Zhang et al., 2008) are also commonly used benchmarks, and provide two types of difficulties. For CF1-CF3 and CF8-CF10, the PFs are parts of their unconstrained PFs. The rest of the CF test problems CF4-CF7 have difficulties near their PFs, and many constrained Pareto-optimal points lie on constraint boundaries.

Even though CTP (Deb, 2001) and CF (Zhang et al., 2008) offer the above-mentioned advantages, they have some limitations:

• The difficulty level of each type is not adjustable.

• No constraint functions with low ratios of feasible regions in the entire search space are suggested.

• The number of objectives is not scalable.

Other sometimes used two-objective test problems include BNH (Binh and Korn, 1997), TNK (Tanaka et al., 1995), SRN (Srinvas and Deb, 1994), and OSY (Osyczka and Kundu, 1995) problems, which are not scalable in the number of objectives, and in which it is harder to identify their types of difficulties.

In this article, we propose a general framework to construct difficulty adjustable and objective scalable CMOPs which can overcome the limitations of existing CMOPs. CMOPs constructed by this toolkit can be classified into three major types, which are feasibility-hard, convergence-hard, and diversity-hard CMOPs. A feasibility-hard CMOP is a type of problem that makes it difficult for CMOEAs to find feasible solutions in the search space. CMOPs with feasibility-hardness usually have small portions of the entire search space that are feasible regions. In contrast, CMOPs with convergence-hardness mainly make it difficult for CMOEAs to approach the PFs efficiently, by setting many infeasibility obstacles before the PFs. CMOPs with diversity-hardness mainly make it difficult for CMOEAs to distribute their solutions along the complete PFs. In our work, any or all three types of difficulty may be embedded into CMOPs through proper construction of constraint functions.

Of course, one interested in solving a particular CMOP must be cautious not to put too much reliance in the performance of a given algorithm on any set of benchmark problems. It would be extremely useful if there were a technique to characterize an arbitrary real-world problem, with its generally long evaluation time, according to a set of characteristics that then could be captured in benchmark problems that could be evaluated much more quickly to compare search algorithms. Unfortunately, no such function characterization method exists, so selecting a CMOEA must be based on less formal understanding of the problem characteristics. The No-Free-Lunch Theorem (Wolpert and Macready, 1997) establishes that different algorithms are better suited to optimizing different sorts of functions, so it is useful to benchmark algorithms on problems that are as similar as possible to the one which is actually being solved. Since there is no analytical way to do this, it appears that having a set of benchmark functions with tunable levels of difficulty of different sorts is at least a step in a productive direction.

In summary, the contributions of this article are as follows:

1. This article defines three primary types of difficulty for constraints in CMOPs. When designing new constraint handling mechanisms for a CMOEA, one has to investigate the nature of constraints in a CMOP that the CMOEA is aiming to address, including the types and levels of difficulties embedded in the constraints. Therefore, the ability to define arbitrary amounts of multiple types of difficulty for constraints in CMOPs is necessary and desirable.

2. This article also defines a level of difficulty for each type of difficulty for constraints in the constructed CMOPs, which can be adjusted by users. A difficulty level is uniquely defined by a triplet with each of its parameters specifying the level of one primary difficulty type. Combination of the three primary constraint types with different difficulty triplets can lead to construction of a large variety of CMOPs.

3. Based on the proposed three primary types of difficulty for constraints, nine difficulty adjustable and scalable CMOPs and CMaOPs, called DAS-CMOP1-9 and DAS-CMaOP1-9, have been constructed.

The remainder of this article is organized as follows. Section 2 discusses the effects of constraints on PFs. Section 3 introduces the types and levels of difficulty provided by constraints in CMOPs. Section 4 explains the proposed toolkit of construction methods for generating constraints in CMOPs with different types and levels of difficulty. Section 5 specifies the scalability of the number of objectives in CMOPs using the proposed toolkit. Section 6 generates a set of difficulty adjustable CMOPs using the proposed toolkit. In Section 7, the performance of two CMOEAs on DAS-CMOP1-9 with different difficulty levels is compared by experimental studies, and Section 8 concludes the article.

## 2  Effects of Constraints on PFs

Constraints define the infeasible regions in the search space, leading to different types and levels of difficulty for the resulting CMOPs. Some major possible effects of the constraints on the PFs in CMOPs include the following (Jain and Deb, 2014):

1. Infeasible regions make the original unconstrained PF only partially feasible. This can be further divided into two situations. In the first situation, the PF of the constrained problem consists of a part of its unconstrained PF and a set of solutions on some boundaries of constraints, as illustrated by Figure 1a. In the second situation, the PF of the constrained problem is only a portion of its unconstrained PF, as illustrated by Figure 1b.

2. Infeasible regions block the way towards the PF, as illustrated by Figure 1c.

3. The complete original PF (of the unconstrained problem) is covered by infeasible regions and is no longer feasible. Every constrained Pareto optimal point lies on a constraint boundary, as illustrated by Figure 1d.

4. Constraints may reduce the dimensionality of the PF, with one example illustrated by Figure 1e. In general, although the problem is $M-$dimensional, constraints can make the constrained PF$K-$dimensional (where $K). In the particular case of Figure 1e, $M=2,K=1$.

Figure 1:

Illustration of the effects of constraints on PFs. (a) Infeasible regions make the original unconstrained PF partially feasible. Many constrained Pareto optimal solutions lie on constraint boundaries. (b) Infeasible regions make the original unconstrained PF partially feasible. The constrained PF is a portion of its unconstrained PF. (c) Infeasible regions block the way toward converging to the PF. The constrained PF is same to its unconstrained PF. (d) The complete original PF is no longer feasible. Every constrained Pareto optimal solution lies on a constraint boundary. (e) Constraints reduce the dimensionality of the PF. A two-objective optimization problem is transformed into a constrained single-objective optimization problem.

Figure 1:

Illustration of the effects of constraints on PFs. (a) Infeasible regions make the original unconstrained PF partially feasible. Many constrained Pareto optimal solutions lie on constraint boundaries. (b) Infeasible regions make the original unconstrained PF partially feasible. The constrained PF is a portion of its unconstrained PF. (c) Infeasible regions block the way toward converging to the PF. The constrained PF is same to its unconstrained PF. (d) The complete original PF is no longer feasible. Every constrained Pareto optimal solution lies on a constraint boundary. (e) Constraints reduce the dimensionality of the PF. A two-objective optimization problem is transformed into a constrained single-objective optimization problem.

## 3  Difficulty Types and Levels of CMOPs

In this section, three primary difficulty types, including convergence-hardness, diversity-hardness, and feasibility-hardness, are illustrated. The difficulty level for each primary difficulty type is also described.

### 3.1  Difficulty 1: Diversity-Hardness

Generally, the PFs of CMOPs with diversity-hardness have many discrete segments, or some parts that are more difficult to achieve than other parts, because large infeasible regions are imposed in their vicinity. As a result, achieving the complete PF is difficult for CMOEAs.

### 3.2  Difficulty 2: Feasibility-Hardness

For feasibility-hard CMOPs, the proportion of feasible regions in the search space is usually very low. It is difficult for a CMOEA to find any feasible solutions on CMOPs with feasibility-hardness. Often in the initial stage of a CMOEA, most or all solutions in the population are infeasible.

### 3.3  Difficulty 3: Convergence-Hardness

CMOPs with convergence-hardness hinder the convergence of CMOEAs toward the PFs. Usually, CMOEAs encounter more difficulty in achieving the PFs because infeasible regions block the way as they converge toward the PFs. In other words, the generational distance (GD) metric (Van Veldhuizen and Lamont, 1998), which indicates the convergence performance, is difficult to minimize in the evolutionary process.

### 3.4  Difficulty Level of Each Primary Difficulty Type

A difficulty level of each primary difficulty type can be defined by a parameter in the parameterized constraint function corresponding to the primary difficulty type. Each parameter is normalized from 0 to 1. Three parameters, corresponding to the difficulty levels of the three primary difficulty types, form a triplet $(η,ζ,γ)$ that exactly defines the difficulty signature of a CMOP constructed by the three parameterized constraint functions. If we allow the three parameters to take any value between 0 and 1, then we can literally get countless varieties of difficulty (analogous to countless colors in the color space, as shown in Figure 2). A difficulty signature is then precisely depicted by a triplet $(η,ζ,γ)$.

Figure 2:

Illustration of difficulty types and levels. (a) The three primary difficulty types and their combinations resulting in seven basic difficulty types (as shown in Table 1), using the analogy with three primary colors and their combinations forming seven basic colors. (b) Combining three parameterized constraint functions using a triplet composed of three parameters. The three primary constraint functions correspond to the three primary difficulty types.

Figure 2:

Illustration of difficulty types and levels. (a) The three primary difficulty types and their combinations resulting in seven basic difficulty types (as shown in Table 1), using the analogy with three primary colors and their combinations forming seven basic colors. (b) Combining three parameterized constraint functions using a triplet composed of three parameters. The three primary constraint functions correspond to the three primary difficulty types.

Table 1:
Basic difficulty types of CMOPs.
Basic Difficulty TypesComment
T1: Diversity-hardness Distributing the feasible solutions in the complete PF is difficult.
T2: Feasibility-hardness Obtaining a feasible solution is difficult.
T3: Convergence-hardness Approaching a Pareto optimal solution is difficult.
T4: Diversity-hardness and feasibility-hardness Obtaining a feasible solution and the complete PF is difficult.
T5: Diversity-hardness and convergence-hardness Approaching a Pareto optimal solution and the complete PF is difficult.
T6: Feasibility-hardness and convergence-hardness Obtaining a feasible solution and approaching a Pareto optimal solution is difficult.
T7: Diversity-hardness, feasibility-hardness, and convergence-hardness Obtaining a Pareto optimal solution and the complete PF is difficult.
Basic Difficulty TypesComment
T1: Diversity-hardness Distributing the feasible solutions in the complete PF is difficult.
T2: Feasibility-hardness Obtaining a feasible solution is difficult.
T3: Convergence-hardness Approaching a Pareto optimal solution is difficult.
T4: Diversity-hardness and feasibility-hardness Obtaining a feasible solution and the complete PF is difficult.
T5: Diversity-hardness and convergence-hardness Approaching a Pareto optimal solution and the complete PF is difficult.
T6: Feasibility-hardness and convergence-hardness Obtaining a feasible solution and approaching a Pareto optimal solution is difficult.
T7: Diversity-hardness, feasibility-hardness, and convergence-hardness Obtaining a Pareto optimal solution and the complete PF is difficult.

## 4  Construction Toolkit

As we know, constructing a CMOP involves constructing two major parts—objective functions and constraint functions. Li et al. (2014) suggested a general framework for constructing objective functions for benchmark problems. It is stated as follows:
$fi(x)=αi(x1:m-1)+βi(x1:m-1,xm:n),$
(2)
where $x1:m-1=(x1,…,xm-1)T,xm:n=(xm,…,xn)T$ are two subvectors of $x=(x1,…,xn)T$. The function $αi(x1:m-1)$ is called the shape function, and $βi(x1:m-1,xm:n)$ is called the nonnegative distance function. The objective function $fi(x),i=1,…,m$ is the sum of the shape function $αi(x1:m-1)$ and the nonnegative distance function $βi(x1:m-1,xm:n)$. We adopt the method of Li et al. (2014) to construct objective functions in this work.

In terms of constructing the constraint functions, three different types of constraint functions are suggested in this article, corresponding to the three proposed primary types of difficulties of CMOPs. More specifically, Type-I constraint functions provide difficulty of diversity-hardness, Type-II constraint functions introduce difficulty of feasibility-hardness, and Type-III constraint functions generate difficulty of convergence-hardness. The detailed definitions of the three types of constraint functions are given as follows:

### 4.1  Type-I Constraint Functions: Diversity-Hardness

Type-I constraint functions are defined to limit the boundaries of sub-vector $x1:m-1$. More specifically, this type of constraint function divides the PF of a CMOP into a number of disconnected segments, generating difficulty of diversity-hardness. Here, we use a parameter $η∈[0,1]$ to represent the level of difficulty. $η=0$ means the constraint functions impose no effect on the CMOP, while $η=1$ means the constraint functions provide their maximum effect.

An example of a CMOP with diversity-hardness is suggested as follows:
$minimizef1(x)=x1+g(x)minimizef2(x)=1-x12+g(x)g(x)=∑i=2n(xi-sin(0.5πx1))2subjecttoc(x)=sin(aπx1)-b≥0xi∈[0,1],$
(3)
where $a>0$, $b∈[-1,1]$. As an example, $a=10$ and $n=2$ are set here. The parameter $η$ specifying the level of difficulty determines $b=2η-1$. The number of disconnected segments in the PF is controlled by $a$. Moreover, the value of $b$ controls the width of each segment. The width of each segment is at its maximum when $b=-1$ (and $η=0$). When $b$ increases, the width of each segment decreases, and the difficulty level increases, and so does the parameter of the difficulty level $η$. As a result, if $η$ is set to 0.55, the PF is shown in Figure 3a. If $η=0.75$, the PF appears as shown in Figure 3b. It can be observed that the width of each segment of the PF is reduced as $η$ keeps increasing. If $η=1.0$, the width of each segment shrinks to zero as shown in Figure 3c, which provides the maximum level of difficulty to the CMOP. The PF of a three-objective CMOP with Type-I constraint functions is also shown in Figure 3d, with the difficult level $η=0.75$. It can be seen that Type-I constraint functions can be applied in more than two-objective CMOPs, which means that a CMOP with a scalable number of objectives can be constructed using this type of constraint function.
Figure 3:

Illustrations of the influence of Type-I constraint functions. When the parameter of difficulty level $η$ increases, the width of each segment in the PF decreases, and the difficulty level of a CMOP increases. The PF of a CMOP with Type-I constraint functions is disconnected and usually has many discrete segments, so obtaining the complete PF is difficult. Thus a CMOP with Type-I constraint functions is diversity-hard. (a) A two-objective CMOP with $η=0.55$. (b) A two-objective CMOP with $η=0.75$. (c) A two-objective CMOP with $η=1.0$. (d) A three-objective CMOP with $η=0.75$.

Figure 3:

Illustrations of the influence of Type-I constraint functions. When the parameter of difficulty level $η$ increases, the width of each segment in the PF decreases, and the difficulty level of a CMOP increases. The PF of a CMOP with Type-I constraint functions is disconnected and usually has many discrete segments, so obtaining the complete PF is difficult. Thus a CMOP with Type-I constraint functions is diversity-hard. (a) A two-objective CMOP with $η=0.55$. (b) A two-objective CMOP with $η=0.75$. (c) A two-objective CMOP with $η=1.0$. (d) A three-objective CMOP with $η=0.75$.

### 4.2  Type-II Constraint Functions: Feasibility-Hardness

Type-II constraint functions are set to limit the reachable boundaries of the distance function of $βi(x1:m-1,xm:n)$, and thereby control the proportion of the search space that is feasible. That is, they generate the difficulty of feasibility-hardness. Here, we use a parameter $ζ$ to represent the level of difficulty, which ranges from 0 to 1. $ζ=0$ means the constraints are the weakest, and $ζ=1.0$ means the constraint functions are the strongest.

For example, a CMOP with Type-II constraint functions can be defined as follows:
$minimizef1(x)=x1+g(x)minimizef2(x)=1-x12+g(x)g(x)=∑i=2n(xi-sin(0.5πx1))2subjecttoc1(x)=g(x)-a≥0c2(x)=b-g(x)≥0n=30,xi∈[0,1],$
(4)
where $ζ$ equals to $1exp(b-a)$, $a≥0$, $b≥0$ and $b≥a$. The distance between the constrained PF and unconstrained PF is controlled by $a$, and $a=0.5$ in this example. The proportion of feasible regions is controlled by $b-a$. For an arbitrary $a$, $b$ must be set to $a-lnζ$ to achieve a hardness in (0,1]. As $b-a$ approaches $+∞$, $ζ$ approaches 0, and the feasible area grows, as shown in Figure 4a. If $ζ=0.905$, $b-a=0.1$, and the feasible area is decreased as shown in Figure 4b. For $ζ=1.0$, $b=a$, and the feasible area in the objective space is very small, which can be observed in Figure 4c. Type-II constraints can be also applied to CMOPs with three objectives, as shown in Figure 4d.
Figure 4:

Illustrations on the influence of Type-II constraint functions. The parameter of difficulty degree $ζ=1exp(b-a)$ determines the proportion of feasible regions. Here, the value of $a$, the offset of the constrained PF from the unconstrained PF, is defaulted to 0.5. When parameter $ζ$ increases, the proportion of feasible area decreases, and the difficulty level of feasibility increases. (a) $ζ=0.0$. (b) $ζ=0.905$. (c) $ζ=1.0$. (d) Type-II constraint functions can also be applied in three-objective optimization problems, as shown with $ζ=0.905$.

Figure 4:

Illustrations on the influence of Type-II constraint functions. The parameter of difficulty degree $ζ=1exp(b-a)$ determines the proportion of feasible regions. Here, the value of $a$, the offset of the constrained PF from the unconstrained PF, is defaulted to 0.5. When parameter $ζ$ increases, the proportion of feasible area decreases, and the difficulty level of feasibility increases. (a) $ζ=0.0$. (b) $ζ=0.905$. (c) $ζ=1.0$. (d) Type-II constraint functions can also be applied in three-objective optimization problems, as shown with $ζ=0.905$.

### 4.3  Type-III Constraint Functions: Convergence-Hardness

Type-III constraint functions limit the reachable boundary of objectives. As a result, infeasible regions act like “blocking” obstacles for the population of a CMOEA searching for the PF. As a result, Type-III constraint functions generate the difficulty of convergence-hardness. Here, we use a parameter $γ$ to represent the level of difficulty, which ranges from 0 to 1. $γ=0$ means the constraints are the weakest, $γ=1$ means the constraints are the strongest, and the difficulty level increases as $γ$ increases.

For example, a CMOP with Type-III constraint functions can be defined as follows:
$minf1(x)=x1+g(x)minf2(x)=1-x12+g(x)whereg(x)=∑i=2n(xi-sin(0.5πx1))2s.t.ck(x)=((f1-pk)cosθk-(f2-qk)sinθk)2/ak2+((f1-pk)sinθk+(f2-qk)cosθk)2/bk2≥rpk=[0,1,0,1,2,0,1,2,3]qk=[1.5,0.5,2.5,1.5,0.5,3.5,2.5,1.5,0.5]ak2=0.4,bk2=1.6,θk=-0.25πc=20,n=30,xi∈[0,1],k=1,…,9,$
(5)
where the level of difficulty parameter $γ$ determines parameter $r$ as $r=γ/2$. If $γ=0.1$, the PF is shown in Figure 5a. If $γ=0.5$, the infeasible regions are increased and shown in Figure 5b. If $γ=0.75$, the infeasible regions become bigger than those of $γ=0.5$ as shown in Figure 5c. Type-III constraint functions can be also applied to CMOPs with three objectives, as shown in Figure 5d.
Figure 5:

Illustrations of the influence of Type-III constraint functions. Infeasible regions block the way of converging to the PF. The gray parts of each figure are infeasible regions. A parameter $γ$ is adopted to represent the level of difficulty, which ranges from 0 to 1. $γ=0$ means the constraints are the weakest, and $γ=1$ means the constraints are the strongest. When $γ$ increases, the difficulty level of convergence-hardness of a CMOP increases. (a) $γ=0.1$. (b) $γ=0.5$. (c) $γ=0.75$. (d) Type-III constraint functions can also be applied in three-objective optimization problems, as shown here with $γ=0.5$.

Figure 5:

Illustrations of the influence of Type-III constraint functions. Infeasible regions block the way of converging to the PF. The gray parts of each figure are infeasible regions. A parameter $γ$ is adopted to represent the level of difficulty, which ranges from 0 to 1. $γ=0$ means the constraints are the weakest, and $γ=1$ means the constraints are the strongest. When $γ$ increases, the difficulty level of convergence-hardness of a CMOP increases. (a) $γ=0.1$. (b) $γ=0.5$. (c) $γ=0.75$. (d) Type-III constraint functions can also be applied in three-objective optimization problems, as shown here with $γ=0.5$.

Type-III constraint functions can be expressed in a matrix form, which can be defined as follows:
$(F(x)-Hk)TSk(F(x)-Hk)≥r,$
(6)
where $F(x)=(f1(x),…,fm(x))T$. $Hk$ is a translation vector. $Sk$ is a transformation matrix that controls the degree of rotation and stretching of the vector $(F(x)-Hk)$. According to the Type-III constraint functions in Eq. (5), $Hk=(pk,qk)T$, and $Sk$ can be expressed as follows:
$Sk=cos2θkak2+sin2θkbk2-sin2θkak2sin2θkbk2cos2θkbk2+sin2θkak2.$

To summarize, the three types of constraint functions discussed above correspond to the three primary difficulty types of CMOPs. The level of each primary difficulty type can be decided by a parameter. In this work, the three parameters are defined in a triplet $(η,ζ,γ)$, which specifies the difficulty levels of the various primary types of difficulty. This approach of constructing a toolkit for CMOPs can also be scaled to generate CMOPs with more than three objective functions. The scalability to the number of objectives is discussed in more detail next.

## 5  Scalability of the Number of Objectives

Many-objective optimization has attracted much research interest in recent years, which makes the scalability of the number of objectives of CMOPs a desirable feature. A general framework to construct CMOPs with a scalable number of objectives is given as follows:
$minf1(x)=α1(x1:m-1)+β1(x1:m-1,xm:n)minf2(x)=α2(x1:m-1)+β2(x1:m-1,xm:n)⋮minfm(x)=αm(x1:m-1)+βm(x1:m-1,xm:n)s.t.ck(x)=sin(aπxk)-b≥0,ifkisoddck(x)=cos(aπxk)-b≥0,ifkisevencK+p(x)=(e-βp)(βp-d)≥0cK+P+q(x)=(F(x)-Hq)TSq(F(x)-Hq)≥rk=1,…,K,andK≤m-1p=1,…,P,andP≤mq=1,…,Qη=(b+1)/2ζ=exp(d-e),d≤e,ifζ==0,d=0γ=2r.$
(7)

In Eq. (7), we borrow an idea from the WFG toolkit (Huband et al., 2006) to construct objectives, which can be scaled to any number of objectives. More specifically, the number of objectives is controlled by a user-defined parameter $m$.

The three different types of constraint functions proposed in Section 5 can be combined with the scalable objective functions to construct difficulty adjustable and scalable CMOPs (DAS-CMOPs). More specifically, the first $K$ constraint functions of Type-I are defined to limit the reachable boundary of each decision variable in the shape functions ($α1(x1:m-1)$ to $αm(x1:m-1)$), which have the ability to control the difficulty level of diversity-hardness using $η$. The $(K+1)$ to $(K+P)$ constraint functions belong to Type-II, which limit the reachable boundary of the distance functions ($β1(x1:m-1,xm:n)$ to $β1(x1:m-1,xm:n)$). They have the ability to control the difficulty level of feasibility-hardness using $ζ$. The last $Q$ constraint functions are set directly on each objective, and belong to Type-III. They generate a number of infeasible regions, which hinder the working population of a CMOEA as it approaches the PF. The difficulty level of convergence-hardness generated by Type-III constraint functions is controlled by $γ$. The other parameters in Eq. (7) are illustrated as follows.

Three parameters—$K$, $P$, and $Q$—are used to control the number of each type of constraint function. $K≤m-1,P≤m$ and $Q≥1$. The total number of constraint functions is controlled by $(K+P+Q)$. $n$ determines the dimension of decision variables, and $n≥m$. $a$ decides the number of disconnected segments in a PF. $d$ indicates the distance between the constrained PF and the unconstrained PF.

It is worth noting that the number of objectives of DAS-CMOPs can be easily scaled by tuning parameter $m$. The difficulty level of DAS-CMOPs can be also easily adjusted by assigning a difficulty triplet $(η,ζ,γ)$ with three parameters ranging from 0 to 1.

## 6  A Set of Difficulty-Adjustable and Scalable CMOPs

In this section, as an example, a set of nine difficulty adjustable and scalable CMOPs (DAS-CMOP1-9) and a set of nine difficulty adjustable and scalable CMaOPs (DAS-CMaOP1-9) are suggested using the proposed toolkit.

As mentioned in Section 4, constructing a CMOP includes constructing both objective functions and constraint functions. According to Eq. (7), we suggest nine multiobjective functions, including convex, concave, and discrete PF shapes, to construct CMOPs. A set of difficulty adjustable constraint functions is generated by Eq. (7). Nine difficulty adjustable and scalable CMOPs, called DAS-CMOP1-9, are generated by combining the suggested objective functions and the generated constraint functions. The detailed definitions of DAS-CMOP1-9 are given in Table 2.

Table 2:
DAS-CMOPs Test suite: The objective functions and constraint functions of DAS-CMOP1-9.
ProblemObjectivesConstraints
DAS-CMOP1 $minf1(x)=x1+g(x)minf2(x)=1-x12+g(x)whereg(x)=∑j=1n(xj-sin(0.5πx1))2n=30,x∈[0,1]n$ $c1(x)=sin(aπx1)-b≥0c2(x)=(e-g(x))(g(x)-d)≥0ck+2(x)=((f1-pk)cosθk-(f2-qk)sinθk)2/ak2+((f1-pk)sinθk+(f2-qk)cosθk)2/bk2≥rpk=[0,1,0,1,2,0,1,2,3],ak2=0.3,bk2=1.2,θk=-0.25πqk=[1.5,0.5,2.5,1.5,0.5,3.5,2.5,1.5,0.5]a=20,d=0.5,η=(b+1)/2,ζ=exp(d-e),γ=2r$
DAS-CMOP2 $minf1(x)=x1+g(x)minf2(x)=1-x1+g(x)whereg(x)=∑j=1n(xj-sin(0.5πx1))2n=30,x∈[0,1]n$ They are the same as those of DAS-CMOP1
DAS-CMOP3 $minf1(x)=x1+g(x)minf2(x)=1-x1+0.5*|sin(5πx1)|+g(x)whereg(x)=∑j=1n(xj-sin(0.5πx1))2n=30,x∈[0,1]n$ They are the same as those of DAS-CMOP1
DAS-CMOP4 $minf1(x)=x1+g(x)minf2(x)=1-x12+g(x)whereg(x)=(n-1)+∑j=2n(xj-0.5)2-cos(20π(xj-0.5))n=30,x∈[0,1]n$ $c1(x)=sin(aπx1)-b≥0c2(x)=(e-g(x))(g(x)-d)≥0ck+2(x)=((f1-pk)cosθk-(f2-qk)sinθk)2/ak2+((f1-pk)sinθk+(f2-qk)cosθk)2/bk2≥rpk=[0,1,0,1,2,0,1,2,3],ak2=0.3,bk2=1.2,θk=-0.25πqk=[1.5,0.5,2.5,1.5,0.5,3.5,2.5,1.5,0.5]a=20,d=0.5,η=(b+1)/2,ζ=exp(d-e),γ=2r$
DAS-CMOP5 $minf1(x)=x1+g(x)minf2(x)=1-x1+g(x)whereg(x)=(n-1)+∑j=2n(xj-0.5)2-cos(20π(xj-0.5))n=30,x∈[0,1]n$ They are the same as those of DAS-CMOP4
DAS-CMOP6 $minf1(x)=x1+g(x)minf2(x)=1-x1+0.5*|sin(5πx1)|+g(x)whereg(x)=(n-1)+∑j=2n(xj-0.5)2-cos(20π(xj-0.5))n=30,x∈[0,1]n$ They are the same as those of DAS-CMOP4
DAS-CMOP7 $minf1(x)=x1*x2+g(x)minf2(x)=x2*(1-x1)+g(x)minf3(x)=1-x2+g(x)whereg(x)=(n-2)+∑j=3n(xj-0.5)2-cos(20π(xj-0.5))n=30,x∈[0,1]n$ $c1(x)=sin(aπx1)-b≥0c2(x)=cos(aπx2)-b≥0c3(x)=(e-g(x))(g(x)-d)≥0ck+3(x)=∑j=1,j≠k3fj2+(fk-1)2-r2≥0c7(x)=∑j=13(fj-13)2-r2≥0a=20,d=0.5,k=1,2,3η=(b+1)/2,ζ=exp(d-e),γ=2r$
DAS-CMOP8 $minf1(x)=cos(0.5πx1)*cos(0.5πx2)+g(x)minf2(x)=cos(0.5πx1)*sin(0.5πx2)+g(x)minf3(x)=sin(0.5πx1)+g(x)whereg(x)=(n-2)+∑j=3n(xj-0.5)2-cos(20π(xj-0.5))n=30,x∈[0,1]n$ They are the same as those of DAS-CMOP7
DAS-CMOP9 $minf1(x)=cos(0.5πx1)*cos(0.5πx2)+g(x)minf2(x)=cos(0.5πx1)*sin(0.5πx2)+g(x)minf3(x)=sin(0.5πx1)+g(x)whereg(x)=∑j=3n(xj-cos(0.25jnπ(x1+x2)))2n=30,x∈[0,1]n$ They are the same as those of DAS-CMOP7
ProblemObjectivesConstraints
DAS-CMOP1 $minf1(x)=x1+g(x)minf2(x)=1-x12+g(x)whereg(x)=∑j=1n(xj-sin(0.5πx1))2n=30,x∈[0,1]n$ $c1(x)=sin(aπx1)-b≥0c2(x)=(e-g(x))(g(x)-d)≥0ck+2(x)=((f1-pk)cosθk-(f2-qk)sinθk)2/ak2+((f1-pk)sinθk+(f2-qk)cosθk)2/bk2≥rpk=[0,1,0,1,2,0,1,2,3],ak2=0.3,bk2=1.2,θk=-0.25πqk=[1.5,0.5,2.5,1.5,0.5,3.5,2.5,1.5,0.5]a=20,d=0.5,η=(b+1)/2,ζ=exp(d-e),γ=2r$
DAS-CMOP2 $minf1(x)=x1+g(x)minf2(x)=1-x1+g(x)whereg(x)=∑j=1n(xj-sin(0.5πx1))2n=30,x∈[0,1]n$ They are the same as those of DAS-CMOP1
DAS-CMOP3 $minf1(x)=x1+g(x)minf2(x)=1-x1+0.5*|sin(5πx1)|+g(x)whereg(x)=∑j=1n(xj-sin(0.5πx1))2n=30,x∈[0,1]n$ They are the same as those of DAS-CMOP1
DAS-CMOP4 $minf1(x)=x1+g(x)minf2(x)=1-x12+g(x)whereg(x)=(n-1)+∑j=2n(xj-0.5)2-cos(20π(xj-0.5))n=30,x∈[0,1]n$ $c1(x)=sin(aπx1)-b≥0c2(x)=(e-g(x))(g(x)-d)≥0ck+2(x)=((f1-pk)cosθk-(f2-qk)sinθk)2/ak2+((f1-pk)sinθk+(f2-qk)cosθk)2/bk2≥rpk=[0,1,0,1,2,0,1,2,3],ak2=0.3,bk2=1.2,θk=-0.25πqk=[1.5,0.5,2.5,1.5,0.5,3.5,2.5,1.5,0.5]a=20,d=0.5,η=(b+1)/2,ζ=exp(d-e),γ=2r$
DAS-CMOP5 $minf1(x)=x1+g(x)minf2(x)=1-x1+g(x)whereg(x)=(n-1)+∑j=2n(xj-0.5)2-cos(20π(xj-0.5))n=30,x∈[0,1]n$ They are the same as those of DAS-CMOP4
DAS-CMOP6 $minf1(x)=x1+g(x)minf2(x)=1-x1+0.5*|sin(5πx1)|+g(x)whereg(x)=(n-1)+∑j=2n(xj-0.5)2-cos(20π(xj-0.5))n=30,x∈[0,1]n$ They are the same as those of DAS-CMOP4
DAS-CMOP7 $minf1(x)=x1*x2+g(x)minf2(x)=x2*(1-x1)+g(x)minf3(x)=1-x2+g(x)whereg(x)=(n-2)+∑j=3n(xj-0.5)2-cos(20π(xj-0.5))n=30,x∈[0,1]n$ $c1(x)=sin(aπx1)-b≥0c2(x)=cos(aπx2)-b≥0c3(x)=(e-g(x))(g(x)-d)≥0ck+3(x)=∑j=1,j≠k3fj2+(fk-1)2-r2≥0c7(x)=∑j=13(fj-13)2-r2≥0a=20,d=0.5,k=1,2,3η=(b+1)/2,ζ=exp(d-e),γ=2r$
DAS-CMOP8 $minf1(x)=cos(0.5πx1)*cos(0.5πx2)+g(x)minf2(x)=cos(0.5πx1)*sin(0.5πx2)+g(x)minf3(x)=sin(0.5πx1)+g(x)whereg(x)=(n-2)+∑j=3n(xj-0.5)2-cos(20π(xj-0.5))n=30,x∈[0,1]n$ They are the same as those of DAS-CMOP7
DAS-CMOP9 $minf1(x)=cos(0.5πx1)*cos(0.5πx2)+g(x)minf2(x)=cos(0.5πx1)*sin(0.5πx2)+g(x)minf3(x)=sin(0.5πx1)+g(x)whereg(x)=∑j=3n(xj-cos(0.25jnπ(x1+x2)))2n=30,x∈[0,1]n$ They are the same as those of DAS-CMOP7

In Table 2, DAS-CMOP1-3 have the same constraint functions. DAS-CMOP4-6 also use that same set of constraint functions. The difference between DAS-CMOP1-3 and DAS-CMOP4-6 is that they have different distance functions. The number of objectives in Eq. (7) can be scaled to more than two. For example, DAS-CMOP7-9 have three objectives. The constraint functions of DAS-CMOP8 and DAS-CMOP9 are the same as those of DAS-CMOP7. The feasible regions and true PFs (PFs with constraints) of DAS-CMOP1-9 with different difficulty triplets are plotted in Figure 6.

Figure 6:

The feasible regions and PFs with constraints (true PFs) of DAS-CMOP1-9 with different difficulty triplets are plotted. Since DAS-CMOP1-3 and DAS-CMOP4-6 have similar feasible regions and PFs when they have the same difficulty triplets, the difficulty triplets for DAS-CMOP1-3 are set to (0, 0.5, 0.5), and the difficulty triplets for DAS-CMOP4-6 are set to (0.5, 0.5, 0.5). For DAS-CMOP7-9, the difficulty triplets are also set to (0.5, 0.5, 0.5).

Figure 6:

The feasible regions and PFs with constraints (true PFs) of DAS-CMOP1-9 with different difficulty triplets are plotted. Since DAS-CMOP1-3 and DAS-CMOP4-6 have similar feasible regions and PFs when they have the same difficulty triplets, the difficulty triplets for DAS-CMOP1-3 are set to (0, 0.5, 0.5), and the difficulty triplets for DAS-CMOP4-6 are set to (0.5, 0.5, 0.5). For DAS-CMOP7-9, the difficulty triplets are also set to (0.5, 0.5, 0.5).

To construct CMOPs with more than three objectives, we borrow the idea from the WFG toolkit (Huband et al., 2006) to construct objectives, which can be scaled to any number of objectives. A general formulation of DAS-CMaOPs is listed as follows.
$Givenz={z1,…,zk,zk+1,…,zn}Minimizefm=1:M(x)=DxM+Smhm(x1:M-1)wherex={x1,…,xM}subjecttock(x)=sin(aπxk)-b≥0,ifkisoddck(x)=cos(aπxk)-b≥0,ifkisevencM(x)=(e-xM)(xM-d)≥0cM+P(x)=∑j=1,j≠PM(fj(x)/Sj)2+(fP(x)/SP-1)2-r2≥0c2M+1(x)=∑j=1Mfj(x)/Sj-1M2-r2≥0a=20,d=0.5,n=30k=1,…,M-1,P=1,…,Mb=η,ifη=0,c1:M-1(x)=0e-d=10-2ζ,ifζ=0,cM(x)=0r=0.5*γ,ifγ=0,cM+1:2M+1(x)=0,$
(8)

where $z$ is a decision vector, and $x$ is an intermediate vector. There is a mapping between $x$ and $z$, and the details can be found in the literature (Huband et al., 2006). In this article, nine DAS-CMaOPs named DAS-CMaOP1-9 are suggested. The objective functions of DAS-CMaOP1-9 are the same as those of WFG1-9 (Huband et al., 2006). The number of position-related parameters in the decision vector $z$ is set to 10, and the number of distance-related parameters in the decision vector $z$ is set to 20.

## 7  Experimental Study

### 7.1  Experimental Settings

To test the performance of CMOEAs on the DAS-CMOPs, two commonly used CMOEAs (i.e., MOEA/D-CDP and NSGA-II-CDP) are tested on DAS-CMOP1-9 with sixteen different difficulty triplets. The difficulty triplets for each DAS-CMOP are listed in Table 3. To test the performance of CMaOEAs on the DAS-CMaOPs, two popular CMaOEAs (i.e., C-MOEA/DD and C-NSGA-III) are tested on DAS-CMaOP1-9 with the first twelve different difficulty triplets in Table 3.

Table 3:
Sixteen difficulty triplets are set for each DAS-CMOP. The difficulty levels of the feasibility-hardness of the last four difficulty triplets are set to 1.0. In this circumstance, some inequality constraints of DAS-CMOPs are transformed into equality constraints, which can be used to test the performance of MOEA/D-CDP and NSGA-II-CDP in solving DAS-CMOPs with equality constraints. For each DAS-CMaOP, the first twelve difficulty triplets are used. Numbers 1, 5, and 9 represent problems that are diversity-hard. Numbers 2, 6, and 10 represent problems that are feasibility-hard. Numbers 3, 7, and 11 represent problems that are convergence-hard. Numbers 4, 8, and 12 represent problems that simultaneously exhibit all three types of difficulty. Numbers 13–15 represent problems with equality constraints.
No.Difficulty TripletNo.Difficulty TripletNo.Difficulty TripletNo.Difficulty Triplet
(0.25,0.0,0.0) (0.0,0.25,0.0) (0.0,0.0,0.25) (0.25,0.25,0.25)
(0.5,0.0,0.0) (0.0,0.5,0.0) (0.0,0.0,0.5) (0.5,0.5,0.5)
(0.75,0.0,0.0) 10 (0.0,0.75,0.0) 11 (0.0,0.0,0.75) 12 (0.75,0.75,0.75)
13 (0.0,1.0,0.0) 14 (0.5,1.0,0.0) 15 (0.0,1.0,0.5) 16 (0.5,1.0,0.5)
No.Difficulty TripletNo.Difficulty TripletNo.Difficulty TripletNo.Difficulty Triplet
(0.25,0.0,0.0) (0.0,0.25,0.0) (0.0,0.0,0.25) (0.25,0.25,0.25)
(0.5,0.0,0.0) (0.0,0.5,0.0) (0.0,0.0,0.5) (0.5,0.5,0.5)
(0.75,0.0,0.0) 10 (0.0,0.75,0.0) 11 (0.0,0.0,0.75) 12 (0.75,0.75,0.75)
13 (0.0,1.0,0.0) 14 (0.5,1.0,0.0) 15 (0.0,1.0,0.5) 16 (0.5,1.0,0.5)

### 7.2  Test Algorithms

To evaluate the proposed test problems with 2 and 3 objectives, two popular algorithms, NSGA-II (Deb et al., 2002) and MOEA/D (Zhang and Li, 2007), are tested. To deal with constraints, a widely used constraint-handling mechanism called CDP (Deb et al., 2001) is integrated with NSGA-II and MOEA/D. The constrained versions of NSGA-II and MOEA/D are called NSGA-II-CDP and MOEA/D-CDP, respectively. Two popular algorithms for solving problems with more than 3 objectives, including NSGA-III (Deb and Jain, 2014; Jain and Deb, 2014) and MOEA/DD (Li et al., 2015), are tested on the proposed test problems with more than 3 objectives. Their constrained versions are called C-NSGA-III and C-MOEA/DD, respectively. A brief introduction of each algorithm is given below.

1. NSGA-II (Deb et al., 2002): NSGA-II is a widely used dominance-based MOEA. In NSGA-II, a fast nondominated sorting operation is applied on the population. Each individual in the population is assigned to a nondominated rank. Solutions are first selected into the next generation based their nondominated rank until the number of solutions is equal to or greater than the population size. Then, crowding distance is applied to select the rest of the solutions to be included.

2. MOEA/D (Zhang and Li, 2007): MOEA/D is a representative of decomposition-based MOEAs. In MOEA/D, a multiobjective optimization problem is decomposed into a number of scalar optimization subproblems, and these subproblems are optimized simultaneously in a cooperative way.

3. NSGA-III (Deb and Jain, 2014; Jain and Deb, 2014): NSGA-III is an improved version of NSGA-II. It overcomes the drawbacks of NSGA-II for solving problems with more than 3 objectives. In NSGA-III, a set of uniformly distributed weight vectors is used to generate niches, associate, and select elite individuals.

4. MOEA/DD (Li et al., 2015): MOEA/DD is a unified paradigm that combines dominance- and decomposition-based approaches for many-objective optimization. It exploits the merits of both dominance- and decomposition-based approaches to balance the convergence and diversity of populations during the evolutionary process.

The source codes of the proposed test problems, DAS-CMOP1-9 and DAS-CMaOP1-9, and the tested algorithms can be found at the following URL: http://imagelab.stu.edu.cn/Content.aspx?type=content&Content_ID=1310.

The detailed parameters of the algorithms are summarized as follows.

1. Setting for reproduction operators: The mutation probability $Pm=1/n$ ($n$ is the number of decision variables). For the polynomial mutation operator, the distribution index is set to 20. For the simulated binary crossover (SBX) operator, the distribution index is set to 20 for DAS-CMOP1-9, and 30 for DAS-CMaOP1-9. The rate of crossover $CR=0.9$.

2. Population size: For DAS-CMOP1-9, $N=300$. For 5-objective DAS-CMaOP1-9, $N=210$, for 8-objective DAS-CMaOP1-9, $N=156$, and for 10-objective DAS-CMaOP1-9, $N=275$ (Deb and Jain, 2014).

3. Number of runs and stopping condition: For DAS-CMOP1-9, each algorithm runs 30 times independently on each test problem, for each of sixteen different difficulty triplets. The maximum number of function evaluations is 300,000 for DAS-CMOP1-9. For DAS-CMaOP1-9, each algorithm runs 20 times independently on each test problem, for each of twelve different difficulty triplets. For 5-, 8-, and 10-objective DAS-CMaOP1-9, the maximum numbers of function evaluations are 400,000, 500,000, and 600,000, respectively.

4. Neighborhood size: $T=⌊0.1N⌋$.

5. Probability of selecting individuals from its neighborhood: $δ=0.9$.

6. The maximal number of solutions replaced by a child: $nr=2$.

### 7.3  Performance Metrics

To measure the performance of MOEA/D-CDP and NSGA-II-CDP on DAS-CMOP1-9 with different difficulty triplets, four different performance metrics are adopted, which are defined as follows.

• Inverted Generational Distance ($IGD$) (Coello and Cortés, 2005):

The $IGD$ metric simultaneously reflects the performance of convergence and diversity, and is defined as follows:
$IGD(P*,A)=∑y*∈P*d(y*,A)|P*|d(y*,A)=miny∈A∑i=1m(yi*-yi)2,$
(9)
where $P*$ is the ideal PF set, $A$ is an approximate PF set achieved by an algorithm, and $m$ represents the number of objectives. It is worth noting that smaller values of $IGD$ represent better performance with respect to both diversity and convergence. To construct $P*$, uniform sampling is performed on the unconstrained PF and constraint boundaries. For 2- and 3-objective problems, 1000 and 10000 points are sampled. Then, nondominated sorting is used to select feasible and nondominated solutions to construct $P*$. For problems with more than 3 objectives, a huge number of uniformly distributed points along the PF would be required to calculate the $IGD$ in a reliable manner (Ishibuchi et al., 2015). In this case, only the hypervolume metric is employed to evaluate the performance of the compared algorithms (Yuan et al., 2016).
• Hypervolume ($HV$) (Zitzler and Thiele, 1999):

$HV$ reflects the closeness of the set of nondominated solutions achieved by a CMOEA to the true PF. A larger $HV$ means that the corresponding nondominated set is closer to the true PF.
$HV(S)=VOL⋃x∈S[f1(x),z1r]×…[fm(x),zmr],$
(10)
where $VOL(·)$ denotes the Lebesgue measure, $m$ is the number of objectives, and $zr=(z1r,…,zmr)T$ is a user-defined reference point in the objective space. For each DAS-CMaOP, the reference point $zr$ is set to $(3.0,…,2.0×m+1.0)T$. For DAS-CMaOPs with 5 and 8 objectives, we adopt the WFG algorithm of While et al. (2012) to calculate the exact $HV$. The presented $HV$ values are all normalized to [0, 1] by dividing them by $z=∏i=1mzir$. For DAS-CMaOPs with 10 objectives, a Monte Carlo sampling is applied to approximate the $HV$, and the sampling size is set to 100000.
• Coverage ($C$-metric):

Let $A$ and $B$ be two approximation sets obtained by two different algorithms. $C(A,B)$ defines the fraction of solutions in $B$ that are dominated by at least one solution in $A$ (Zitzler et al., 2003).
$C(A,B)=|{u∈B∣∃ν∈A:ν≺u}||B|.$
(11)
$C(A,B)=1$ means that each solution in $B$ is dominated by at least one solution in $A$, while $C(A,B)=0$ indicates that no solutions in $B$ are dominated by solutions in $A$.
• Spacing ($Sp$):

$Sp$ is designed to measure how evenly the solutions of an approximation set are distributed. The definition of $Sp$ is as follows:
$Sp(A)=1|A|-1∑x∈A(d¯-dx)2dx=minx*≠xx*∈A∑i=1m|fi(x)-fi(x*)|d¯=1|A|∑x∈Adx.$
(12)
$Sp=0$ means that all solutions of the approximation set are equidistantly spaced.
It is worth noting that only feasible solutions are employed to calculate $IGD$, $HV$, $C$-metric, and $Sp$ values. When dealing with equality constraints, we usually convert the equality constraints into inequality constraints by introducing an extremely small positive number $ε$ as follows (Ullah et al., 2012).
$hj(x)'≡ε-|hj(x)|≥0,$
(13)
where $hj(x)$ is an equality constraint, and $hj(x)'$ is an inequality constraint transformed from $hj(x)$. Here $ε$ is set to $1e-6$. If a solution satisfies Eq. (13), it is considered to be a feasible solution. Otherwise, it is infeasible.

### 7.4  Performance Comparisons on DAS-CMOPs

Table 4 presents the statistics on $IGD$, $C$-metric and spacing values for MOEA/D-CDP and NSGA-II-CDP on DAS-CMOP1-5 with sixteen different difficulty triplets as shown in Table 3. We can see that for DAS-CMOP1 and DAS-CMOP2 with simultaneous diversity-, feasibility-, and convergence-hardness—that is, the difficulty triplets are (0.25, 0.25, 0.25), (0.5, 0.5, 0.5), and (0.75, 0.75, 0.75)—NSGA-II-CDP is significantly better than MOEA/D-CDP in $IGD$ values. For DAS-CMOP1 and DAS-CMOP2 with a single difficulty level, MOEA/D-CDP is significantly better than NSGA-II-CDP in most cases. For DAS-CMOP1 and DAS-CMOP2 with equality constraints—that is, the difficulty triplets are (0.0, 1.0, 0.0), (0.5, 1.0, 0.0), (0.0, 1.0, 0.5), and (0.5, 1.0, 0.5)—NSGA-II-CDP is better or significantly better than MOEA/D-CDP in $IGD$ values.

Table 4:
Means and standard deviations of $IGD$, $C$-metric, and spacing values obtained by MOEA/D-CDP and NSGA-II-CDP on DAS-CMOP1-5. Wilcoxon's rank sum test at 0.05 significance level is performed between MOEA/D-CDP and NSGA-II-CDP. $†$ and $‡$ denote that the performance of NSGA-II-CDP is significantly worse than or better than that of MOEA/D-CDP, respectively. DAS-CMOP1(i) means that DAS-CMOP1 has the $i$-th difficulty triplet in Table 3.
$IGD$$C$-metricSpacing
Test ProblemMOEA/D-CDPNSGA-II-CDPMOEA/D-CDPNSGA-II-CDPMOEA/D-CDPNSGA-II-CDP
DAS-CMOP1(1) 1.29E-03(1.51E-05) 3.70E-01(1.46E-02)$†$ 2.88E-01(9.42E-02) 1.11E-01(1.06E-01)$†$ 2.97E-03(7.20E-05) 1.82E-04(1.33E-04)$‡$
DAS-CMOP1(2) 1.29E-02(4.35E-02) 2.90E-01(1.96E-02)$†$ 1.25E-02(1.78E-02) 1.61E-01(2.69E-02)$‡$ 5.80E-03(1.51E-03) 1.09E-03(3.38E-04)$‡$
DAS-CMOP1(3) 1.30E-03(7.62E-06) 3.77E-01(1.21E-02)$†$ 4.13E-01(1.07E-01) 1.70E-01(1.72E-01)$†$ 1.37E-03(5.54E-06) 2.25E-04(1.39E-04)$‡$
DAS-CMOP1(4) 4.27E-01(8.01E-02) 3.77E-01(2.06E-02)$‡$ 4.50E-02(6.48E-02) 1.50E-01(8.24E-02)$‡$ 1.35E-03(4.35E-04) 5.73E-04(5.36E-04)$‡$
DAS-CMOP1(5) 9.51E-02(1.57E-01) 3.64E-01(1.74E-02)$†$ 1.47E-01(1.34E-01) 1.24E-01(1.50E-01) 2.34E-03(1.32E-03) 1.91E-04(1.52E-04)$‡$
DAS-CMOP1(6) 4.37E-03(2.85E-03) 2.84E-01(1.96E-02)$†$ 2.90E-01(8.34E-02) 1.48E-01(3.01E-02)$†$ 5.61E-03(4.58E-04) 1.10E-03(4.30E-04)$‡$
DAS-CMOP1(7) 1.23E-03(4.32E-06) 3.10E-01(6.86E-03)$†$ 1.64E-01(5.62E-02) 5.94E-02(1.03E-01)$†$ 1.92E-03(3.12E-04) 2.23E-04(1.25E-04)$‡$
DAS-CMOP1(8) 7.28E-01(9.83E-02) 7.09E-01(2.01E-02)$‡$ 1.73E-01(2.96E-01) 1.81E-01(3.42E-02)$‡$ 1.83E-03(6.96E-04) 2.02E-03(2.19E-03)$†$
DAS-CMOP1(9) 3.57E-01(3.60E-06) 3.64E-01(2.08E-02)$†$ 4.05E-01(8.81E-02) 2.14E-01(1.06E-01)$†$ 1.19E-04(2.76E-05) 2.09E-04(4.81E-04)
DAS-CMOP1(10) 5.79E-03(5.00E-03) 2.77E-01(1.88E-02)$†$ 3.26E-01(1.20E-01) 2.06E-01(2.80E-02)$†$ 5.76E-03(1.22E-03) 1.38E-03(7.94E-04)$‡$
DAS-CMOP1(11) 9.82E-02(3.24E-04) 2.42E-01(5.90E-03)$†$ 2.14E-01(7.51E-02) 1.32E-01(1.40E-01)$†$ 1.63E-03(2.69E-04) 2.46E-04(1.44E-04)$‡$
DAS-CMOP1(12) 8.96E-01(4.96E-02) 8.67E-01(2.54E-02)$‡$ 3.33E-02(1.83E-01) 0.00E+00(0.00E+00) 0.00E+00(0.00E+00) 9.12E-08(5.00E-07)$†$
DAS-CMOP1(13) 3.58E-01(2.26E-02) 3.46E-01(2.05E-02)$‡$ 7.75E-04(2.01E-03) 8.89E-03(1.95E-02)$‡$ 7.79E-04(5.38E-04) 9.03E-04(8.73E-04)
DAS-CMOP1(14) 3.61E-01(1.63E-02) 3.43E-01(2.14E-02) 9.70E-03(3.47E-02) 3.02E-02(2.64E-02)$‡$ 4.95E-04(3.00E-04) 6.18E-04(1.17E-03)
DAS-CMOP1(15) 6.79E-01(9.58E-02) 6.28E-01(1.26E-02) 3.33E-02(1.83E-01) 0.00E+00(0.00E+00) 0.00E+00(0.00E+00) 2.56E-05(1.16E-04)$†$
DAS-CMOP1(16) 6.88E-01(1.20E-01) 5.70E-01(1.04E-02) 4.67E-01(5.07E-01) 0.00E+00(0.00E+00)$‡$ 0.00E+00(0.00E+00) 1.29E-06(6.08E-06)$†$
DAS-CMOP2(1) 1.35E-03(9.02E-06) 3.35E-01(1.31E-02)$†$ 2.78E-01(5.03E-02) 1.24E-01(1.29E-01)$†$ 4.60E-03(7.99E-05) 2.94E-04(6.34E-04)$‡$
DAS-CMOP2(2) 4.93E-03(6.35E-03) 2.43E-01(1.93E-02)$†$ 5.83E-01(1.76E-01) 1.57E-01(3.07E-02)$†$ 5.55E-03(6.72E-04) 1.40E-03(6.96E-04)$‡$
DAS-CMOP2(3) 1.32E-03(7.80E-06) 3.36E-01(1.17E-02)$†$ 3.96E-01(9.90E-02) 1.32E-01(1.35E-01)$†$ 3.46E-03(1.17E-04) 2.54E-04(1.56E-04)$‡$
DAS-CMOP2(4) 2.59E-01(4.07E-02) 2.39E-01(1.47E-02)$‡$ 1.26E-02(1.40E-02) 1.38E-01(3.07E-02)$‡$ 1.49E-03(4.07E-04) 1.13E-03(9.84E-04)$‡$
DAS-CMOP2(5) 9.86E-02(1.50E-01) 3.34E-01(1.59E-02)$†$ 1.88E-01(2.02E-01) 1.69E-01(1.30E-01) 3.59E-03(2.28E-03) 2.05E-04(1.13E-04)$‡$
DAS-CMOP2(6) 4.60E-03(6.92E-03) 2.34E-01(1.71E-02)$†$ 3.17E-01(1.32E-01) 1.72E-01(2.57E-02)$†$ 5.53E-03(5.99E-04) 1.21E-03(4.00E-04)$‡$
DAS-CMOP2(7) 1.32E-03(7.01E-06) 3.55E-01(1.95E-02)$†$ 3.10E-01(8.77E-02) 7.02E-02(1.16E-01)$†$ 3.44E-03(1.02E-04) 2.50E-04(1.51E-04)$‡$
DAS-CMOP2(8) 3.39E-01(2.01E-01) 2.16E-01(2.09E-02)$‡$ 8.51E-01(3.48E-01) 1.77E-01(2.48E-02)$†$ 1.22E-03(4.67E-04) 8.93E-04(1.33E-03)$‡$
DAS-CMOP2(9) 3.19E-01(6.00E-02) 3.31E-01(1.37E-02)$†$ 1.51E-02(1.40E-02) 2.65E-01(1.07E-01)$‡$ 2.24E-04(6.22E-04) 1.60E-04(9.21E-05)
DAS-CMOP2(10) 6.21E-03(7.87E-03) 2.35E-01(1.79E-02)$†$ 4.08E-01(1.03E-01) 1.49E-01(2.64E-02)$†$ 5.66E-03(1.36E-03) 1.27E-03(7.24E-04)$‡$
DAS-CMOP2(11) 1.32E-03(6.25E-06) 5.40E-01(2.08E-01)$†$ 2.56E-01(8.50E-02) 6.98E-02(2.44E-01)$†$ 3.46E-03(1.17E-04) 6.89E-04(7.83E-04)$‡$
DAS-CMOP2(12) 4.54E-01(2.59E-01) 2.19E-01(2.43E-02)$‡$ 8.06E-02(1.79E-01) 1.32E-01(3.81E-02)$‡$ 5.93E-04(5.39E-04) 1.37E-03(2.57E-03)
DAS-CMOP2(13) 3.10E-01(2.33E-02) 3.01E-01(1.82E-02) 3.97E-03(8.47E-03) 1.18E-02(1.79E-02)$‡$ 6.22E-04(3.23E-04) 1.44E-03(1.41E-03)$‡$
DAS-CMOP2(14) 3.31E-01(1.70E-02) 3.25E-01(1.86E-02) 1.11E-02(1.86E-02) 3.30E-02(4.03E-02)$‡$ 5.18E-04(3.00E-04) 4.02E-04(3.26E-04)
DAS-CMOP2(15) 3.46E-01(1.16E-02) 3.43E-01(9.25E-03) 3.27E-03(8.68E-03) 1.83E-02(1.82E-02)$†$ 6.69E-04(4.55E-04 6.24E-04(7.77E-04)
DAS-CMOP2(16) 3.12E-01(1.36E-02) 3.09E-01(5.59E-03) 1.13E-02(2.08E-02) 2.88E-02(3.40E-02)$‡$ 4.47E-04(3.20E-04) 3.25E-04(2.39E-04)
DAS-CMOP3(1) 3.10E-02(5.25E-02) 3.37E-01(5.00E-02)$†$ 2.08E-01(7.85E-02) 4.54E-02(1.83E-01)$†$ 3.35E-03(2.82E-04) 1.28E-03(1.31E-03)$‡$
DAS-CMOP3(2) 1.57E-01(3.10E-02) 2.31E-01(1.47E-02)$†$ 2.55E-01(2.26E-01) 1.00E-01(1.51E-01)$†$ 2.99E-03(6.78E-04) 2.89E-03(2.42E-03)$‡$
DAS-CMOP3(3) 2.23E-02(5.23E-02) 2.84E-01(1.50E-02)$†$ 1.35E-01(8.22E-02) 2.48E-01(6.75E-02)$‡$ 3.72E-03(4.87E-04) 6.41E-04(4.40E-04)$‡$
DAS-CMOP3(4) 2.92E-01(8.95E-04) 2.94E-01(4.35E-03)$†$ 7.83E-02(4.14E-02) 4.84E-02(4.46E-02)$†$ 2.25E-03(4.79E-04) 1.71E-03(4.11E-03)$‡$
DAS-CMOP3(5) 9.97E-02(5.29E-02) 4.25E-01(1.06E-01)$†$ 2.51E-01(2.11E-01) 2.00E-01(4.07E-01)$†$ 4.27E-03(8.49E-04) NaN(NaN)
DAS-CMOP3(6) 1.67E-01(2.58E-02) 2.30E-01(1.17E-02)$†$ 3.11E-01(1.67E-01) 4.31E-01(2.30E-01)$‡$ 2.86E-03(5.53E-04) 2.93E-03(2.86E-03)$†$
DAS-CMOP3(7) 3.24E-02(6.16E-02) 2.67E-01(1.83E-02)$†$ 1.30E-01(6.73E-02) 4.31E-01(8.87E-02)$‡$ 3.60E-03(5.69E-04) 9.46E-04(8.74E-04)$‡$
DAS-CMOP3(8) 3.94E-01(1.39E-01) 3.42E-01(2.84E-02) 2.79E-03(4.02E-03) 1.03E-01(9.18E-02)$‡$ 7.94E-04(7.01E-04) 1.05E-03(5.11E-03)$†$
DAS-CMOP3(9) 1.67E-01(8.28E-02) 3.08E-01(4.50E-02)$†$ 2.29E-01(3.51E-01) 3.00E-01(4.66E-01) 1.35E-03(1.68E-03) NaN(NaN)
DAS-CMOP3(10) 1.70E-01(3.54E-02) 2.32E-01(2.10E-02)$†$ 1.85E-01(7.23E-02) 3.35E-01(2.46E-01) 2.81E-03(4.57E-04) 5.11E-03(3.37E-03)
DAS-CMOP3(11) 6.83E-02(9.70E-02) 2.42E-01(2.56E-02)$†$ 2.46E-02(5.21E-02) 2.85E-01(1.15E-01)$‡$ 3.54E-03(8.60E-04) 9.97E-04(9.11E-04)$‡$
DAS-CMOP3(12) 8.28E-01(2.38E-01) 7.95E-01(2.83E-01) 1.33E-01(3.46E-01) 4.70E-02(1.85E-01) 7.14E-05(2.29E-04) 2.47E-05(5.65E-05)
DAS-CMOP3(13) 3.45E-01(7.69E-02) 3.39E-01(9.36E-02) 4.55E-02(8.47E-02) 4.71E-02(7.48E-02) 1.62E-03(1.19E-03) NaN(NaN)
DAS-CMOP3(14) 4.48E-01(3.25E-02) 4.33E-01(4.47E-02) 3.64E-01(3.40E-01) 4.36E-02(2.89E-02) 1.41E-03(7.86E-04) NaN(NaN)
DAS-CMOP3(15) 5.37E-01(2.72E-01) 3.03E-01(3.75E-02)$‡$ 5.74E-02(1.83E-01) 3.43E-02(3.44E-02)$†$ 6.58E-04(7.88E-04) 1.97E-03(2.25E-03)$†$
DAS-CMOP3(16) 6.54E-01(1.69E-01) 4.73E-01(6.34E-02)$‡$ 5.33E-01(5.07E-01) 1.04E-01(3.04E-01)$†$ 3.08E-05(7.80E-05) NaN(NaN)
DAS-CMOP4(1) 5.00E-02(7.13E-02) 1.04E-03(3.40E-05)$‡$ 6.06E-01(4.92E-01) 9.22E-03(3.26E-02)$†$ 3.26E-03(3.91E-04) 1.56E-03(1.05E-04)$‡$
DAS-CMOP4(2) 1.81E-03(6.90E-04) 1.60E-03(9.90E-05) 1.72E-01(4.20E-02) 1.06E-01(1.55E-02)$‡$ 2.32E-03(7.85E-04) 1.73E-03(1.45E-04)$‡$
DAS-CMOP4(3) 8.94E-02(1.78E-01) 1.57E-01(9.91E-02)$†$ 6.77E-02(2.16E-01) 1.36E-01(3.45E-01) 2.37E-03(1.51E-03) 1.00E-02(1.03E-02)$†$
DAS-CMOP4(4) 6.90E-02(6.98E-02) 6.94E-03(2.92E-02)$‡$ 2.78E-01(7.71E-02) 9.08E-02(2.40E-02)$†$ 3.37E-03(5.60E-04) 1.28E-03(2.22E-04)$‡$
DAS-CMOP4(5) 9.66E-02(9.33E-02) 9.91E-04(1.78E-05)$‡$ 8.43E-01(3.22E-01) 7.67E-03(1.89E-02)$†$ 4.05E-03(3.51E-03) 1.14E-03(6.30E-05)$‡$
DAS-CMOP4(6) 1.69E-03(4.57E-04) 1.58E-03(6.20E-05) 6.28E-02(2.89E-02) 1.71E-01(3.13E-02)$‡$ 2.35E-03(5.04E-04) 1.76E-03(1.64E-04)$‡$
DAS-CMOP4(7) 1.13E-01(1.40E-01) 4.90E-01(2.37E-01)$†$ 7.07E-01(3.29E-01) 8.56E-01(3.29E-01)$‡$ 3.28E-03(1.80E-03) 7.73E-03(4.91E-03)$†$
DAS-CMOP4(8) 2.04E-01(2.70E-05) 1.11E-01(1.57E-01)$‡$ 1.12E-01(4.60E-02) 7.98E-02(6.18E-02) 2.95E-03(7.52E-04) 4.01E-03(6.26E-03)$†$
DAS-CMOP4(9) 1.22E-01(1.54E-01) 5.40E-04(3.18E-05)$‡$ 2.15E-01(3.70E-01) 6.11E-03(2.69E-02)$†$ 3.92E-03(3.17E-03) 7.60E-04(4.76E-05)$‡$
DAS-CMOP4(10) 1.48E-03(1.13E-04) 1.57E-03(6.01E-05)$†$ 8.73E-02(2.59E-02) 9.89E-02(1.66E-02)$‡$ 2.29E-03(2.43E-04) 1.72E-03(1.64E-04)$‡$
DAS-CMOP4(11) 2.47E-01(2.30E-01) 8.53E-01(3.27E-01)$†$ 7.46E-01(2.50E-01) 9.51E-02(1.94E-01)$†$ 4.04E-03(2.50E-03) 2.29E-02(9.07E-02)$†$
DAS-CMOP4(12) 2.33E-01(6.95E-05) 2.33E-01(1.48E-04) 9.59E-03(1.11E-02) 1.55E-01(4.84E-02)$‡$ 1.98E-03(1.40E-04) 2.25E-04(3.22E-05)$‡$
DAS-CMOP4(13) 1.44E-03(6.30E-05) 3.79E-03(7.67E-03)$†$ 7.25E-02(3.87E-02) 6.67E-03(5.32E-03)$†$ 2.69E-03(3.65E-04) 2.52E-03(1.03E-03)$‡$
DAS-CMOP4(14) 1.59E-03(9.75E-05) 2.22E-02(6.76E-02)$†$ 1.72E-01(8.13E-02) 6.67E-03(6.06E-03)$†$ 3.56E-03(2.01E-04) 1.63E-03(2.42E-03)$‡$
DAS-CMOP4(15) 2.24E-01(4.07E-05) 2.25E-01(1.23E-03)$†$ 1.12E-01(1.08E-01) 1.11E-03(3.20E-03)$†$ 2.07E-03(3.64E-04) 1.41E-03(2.07E-03)$‡$
DAS-CMOP4(16) 2.72E-01(7.92E-05) 3.02E-01(9.15E-02)$†$ 4.40E-02(3.17E-02) 4.00E-03(2.07E-02)$†$ 2.50E-03(2.13E-04) 6.99E-04(1.27E-03)$‡$
DAS-CMOP5(1) 1.30E-03(8.22E-07) 1.19E-03(7.00E-05)$‡$ 1.66E-01(3.59E-02) 3.67E-03(3.08E-03)$†$ 4.59E-03(3.39E-05) 2.19E-03(2.11E-04)$‡$
DAS-CMOP5(2) 1.95E-03(1.91E-04) 1.48E-03(2.12E-05)$‡$ 3.44E-01(9.58E-02) 9.67E-03(3.54E-03)$†$ 5.19E-03(7.01E-04) 2.28E-03(9.71E-05)$‡$
DAS-CMOP5(3) 1.29E-03(8.64E-07) 1.54E-03(3.93E-05)$†$ 1.70E-01(2.15E-02) 4.67E-03(2.85E-03)$†$ 3.41E-03(6.50E-05) 2.48E-03(2.13E-04)$‡$
DAS-CMOP5(4) 1.93E-03(2.19E-04) 1.08E-03(2.45E-05)$‡$ 3.29E-01(5.88E-02) 1.00E-02(5.81E-03)$†$ 5.35E-03(5.06E-04) 1.89E-03(1.21E-04)$‡$
DAS-CMOP5(5) 1.44E-03(1.42E-06) 1.05E-03(6.39E-05)$‡$ 8.65E-02(5.02E-02) 5.00E-03(3.25E-03)$†$ 4.92E-03(5.17E-05) 1.73E-03(1.83E-04)$‡$
DAS-CMOP5(6) 1.94E-03(3.12E-04) 1.48E-03(2.84E-05)$‡$ 1.70E-01(5.16E-02) 3.00E-03(1.60E-03)$†$ 4.96E-03(6.67E-04) 2.32E-03(1.28E-04)$‡$
DAS-CMOP5(7) 1.29E-03(1.25E-06) 1.52E-03(4.73E-05)$†$ 1.50E-01(2.67E-02) 5.11E-03(4.77E-03)$†$ 3.40E-03(7.56E-05) 2.40E-03(1.27E-04)$‡$
DAS-CMOP5(8) 2.03E-03(1.48E-04) 1.02E-03(2.15E-05)$‡$ 3.64E-01(7.98E-02) 5.86E-02(1.20E-02)$†$ 5.18E-03(3.79E-04) 1.50E-03(8.35E-05)$‡$
DAS-CMOP5(9) 1.22E-03(1.10E-06) 5.94E-04(4.78E-05)$‡$ 3.21E-01(9.32E-02) 2.89E-03(1.15E-03)$†$ 3.50E-03(3.51E-05) 1.10E-03(1.28E-04)$‡$
DAS-CMOP5(10) 1.92E-03(3.83E-04) 1.48E-03(3.41E-05)$‡$ 2.16E-01(3.17E-02) 0.00E+00(0.00E+00)$†$ 4.75E-03(6.92E-04) 2.26E-03(1.37E-04)$‡$
DAS-CMOP5(11) 1.29E-03(8.23E-07) 1.51E-03(4.05E-05)$†$ 1.23E-01(2.95E-02) 8.89E-03(5.20E-03)$†$ 3.41E-03(6.89E-05) 2.41E-03(1.42E-04)$‡$
DAS-CMOP5(12) 1.17E-01(3.15E-01) 1.39E-02(4.24E-04) 1.52E-01(7.85E-02) 7.84E-02(1.26E-02)$†$ 4.03E-03(1.40E-03) 1.05E-03(4.64E-05)$‡$
DAS-CMOP5(13) 1.76E-03(1.91E-04) 1.50E-03(2.98E-05)$‡$ 1.36E-01(6.73E-02) 1.67E-03(5.52E-03)$†$ 4.54E-03(7.31E-04) 2.37E-03(1.23E-04)$‡$
DAS-CMOP5(14) 1.87E-03(2.08E-04) 9.73E-04(2.67E-05)$‡$ 2.91E-01(1.42E-01) 3.78E-03(3.36E-03)$†$ 4.94E-03(3.61E-04) 1.45E-03(9.60E-05)$‡$
DAS-CMOP5(15) 3.02E-01(9.47E-02) 6.26E-02(9.70E-02)$‡$ 1.22E-01(1.16E-01) 2.22E-03(2.53E-03)$†$ 3.54E-03(8.95E-04) 1.50E-03(2.63E-04)$‡$
DAS-CMOP5(16) 3.32E-01(1.04E-01) 6.70E-02(8.60E-02)$‡$ 3.24E-02(7.80E-02) 3.78E-03(3.00E-03)$†$ 2.89E-03(1.01E-03) 8.97E-04(1.00E-04)$‡$
$IGD$$C$-metricSpacing
Test ProblemMOEA/D-CDPNSGA-II-CDPMOEA/D-CDPNSGA-II-CDPMOEA/D-CDPNSGA-II-CDP
DAS-CMOP1(1) 1.29E-03(1.51E-05) 3.70E-01(1.46E-02)$†$ 2.88E-01(9.42E-02) 1.11E-01(1.06E-01)$†$ 2.97E-03(7.20E-05) 1.82E-04(1.33E-04)$‡$
DAS-CMOP1(2) 1.29E-02(4.35E-02) 2.90E-01(1.96E-02)$†$ 1.25E-02(1.78E-02) 1.61E-01(2.69E-02)$‡$ 5.80E-03(1.51E-03) 1.09E-03(3.38E-04)$‡$
DAS-CMOP1(3) 1.30E-03(7.62E-06) 3.77E-01(1.21E-02)$†$ 4.13E-01(1.07E-01) 1.70E-01(1.72E-01)$†$ 1.37E-03(5.54E-06) 2.25E-04(1.39E-04)$‡$
DAS-CMOP1(4) 4.27E-01(8.01E-02) 3.77E-01(2.06E-02)$‡$ 4.50E-02(6.48E-02) 1.50E-01(8.24E-02)$‡$ 1.35E-03(4.35E-04) 5.73E-04(5.36E-04)$‡$
DAS-CMOP1(5) 9.51E-02(1.57E-01) 3.64E-01(1.74E-02)$†$ 1.47E-01(1.34E-01) 1.24E-01(1.50E-01) 2.34E-03(1.32E-03) 1.91E-04(1.52E-04)$‡$
DAS-CMOP1(6) 4.37E-03(2.85E-03) 2.84E-01(1.96E-02)$†$ 2.90E-01(8.34E-02) 1.48E-01(3.01E-02)$†$ 5.61E-03(4.58E-04) 1.10E-03(4.30E-04)$‡$
DAS-CMOP1(7) 1.23E-03(4.32E-06) 3.10E-01(6.86E-03)$†$ 1.64E-01(5.62E-02) 5.94E-02(1.03E-01)$†$ 1.92E-03(3.12E-04) 2.23E-04(1.25E-04)$‡$
DAS-CMOP1(8) 7.28E-01(9.83E-02) 7.09E-01(2.01E-02)$‡$ 1.73E-01(2.96E-01) 1.81E-01(3.42E-02)$‡$ 1.83E-03(6.96E-04) 2.02E-03(2.19E-03)$†$
DAS-CMOP1(9) 3.57E-01(3.60E-06) 3.64E-01(2.08E-02)$†$ 4.05E-01(8.81E-02) 2.14E-01(1.06E-01)$†$ 1.19E-04(2.76E-05) 2.09E-04(4.81E-04)
DAS-CMOP1(10) 5.79E-03(5.00E-03) 2.77E-01(1.88E-02)$†$ 3.26E-01(1.20E-01) 2.06E-01(2.80E-02)$†$ 5.76E-03(1.22E-03) 1.38E-03(7.94E-04)$‡$
DAS-CMOP1(11) 9.82E-02(3.24E-04) 2.42E-01(5.90E-03)$†$ 2.14E-01(7.51E-02) 1.32E-01(1.40E-01)$†$ 1.63E-03(2.69E-04) 2.46E-04(1.44E-04)$‡$
DAS-CMOP1(12) 8.96E-01(4.96E-02) 8.67E-01(2.54E-02)$‡$ 3.33E-02(1.83E-01) 0.00E+00(0.00E+00) 0.00E+00(0.00E+00) 9.12E-08(5.00E-07)$†$
DAS-CMOP1(13) 3.58E-01(2.26E-02) 3.46E-01(2.05E-02)$‡$ 7.75E-04(2.01E-03) 8.89E-03(1.95E-02)$‡$ 7.79E-04(5.38E-04) 9.03E-04(8.73E-04)
DAS-CMOP1(14) 3.61E-01(1.63E-02) 3.43E-01(2.14E-02) 9.70E-03(3.47E-02) 3.02E-02(2.64E-02)$‡$ 4.95E-04(3.00E-04) 6.18E-04(1.17E-03)
DAS-CMOP1(15) 6.79E-01(9.58E-02) 6.28E-01(1.26E-02) 3.33E-02(1.83E-01) 0.00E+00(0.00E+00) 0.00E+00(0.00E+00) 2.56E-05(1.16E-04)$†$
DAS-CMOP1(16) 6.88E-01(1.20E-01) 5.70E-01(1.04E-02) 4.67E-01(5.07E-01) 0.00E+00(0.00E+00)$‡$ 0.00E+00(0.00E+00) 1.29E-06(6.08E-06)$†$
DAS-CMOP2(1) 1.35E-03(9.02E-06) 3.35E-01(1.31E-02)$†$ 2.78E-01(5.03E-02) 1.24E-01(1.29E-01)$†$ 4.60E-03(7.99E-05) 2.94E-04(6.34E-04)$‡$
DAS-CMOP2(2) 4.93E-03(6.35E-03) 2.43E-01(1.93E-02)$†$ 5.83E-01(1.76E-01) 1.57E-01(3.07E-02)$†$ 5.55E-03(6.72E-04) 1.40E-03(6.96E-04)$‡$
DAS-CMOP2(3) 1.32E-03(7.80E-06) 3.36E-01(1.17E-02)$†$ 3.96E-01(9.90E-02) 1.32E-01(1.35E-01)$†$ 3.46E-03(1.17E-04) 2.54E-04(1.56E-04)$‡$
DAS-CMOP2(4) 2.59E-01(4.07E-02) 2.39E-01(1.47E-02)$‡$ 1.26E-02(1.40E-02) 1.38E-01(3.07E-02)$‡$ 1.49E-03(4.07E-04) 1.13E-03(9.84E-04)$‡$
DAS-CMOP2(5) 9.86E-02(1.50E-01) 3.34E-01(1.59E-02)$†$ 1.88E-01(2.02E-01) 1.69E-01(1.30E-01) 3.59E-03(2.28E-03) 2.05E-04(1.13E-04)$‡$
DAS-CMOP2(6) 4.60E-03(6.92E-03) 2.34E-01(1.71E-02)$†$ 3.17E-01(1.32E-01) 1.72E-01(2.57E-02)$†$ 5.53E-03(5.99E-04) 1.21E-03(4.00E-04)$‡$
DAS-CMOP2(7) 1.32E-03(7.01E-06) 3.55E-01(1.95E-02)$†$ 3.10E-01(8.77E-02) 7.02E-02(1.16E-01)$†$ 3.44E-03(1.02E-04) 2.50E-04(1.51E-04)$‡$
DAS-CMOP2(8) 3.39E-01(2.01E-01) 2.16E-01(2.09E-02)$‡$ 8.51E-01(3.48E-01) 1.77E-01(2.48E-02)$†$ 1.22E-03(4.67E-04) 8.93E-04(1.33E-03)$‡$
DAS-CMOP2(9) 3.19E-01(6.00E-02) 3.31E-01(1.37E-02)$†$ 1.51E-02(1.40E-02) 2.65E-01(1.07E-01)$‡$ 2.24E-04(6.22E-04) 1.60E-04(9.21E-05)
DAS-CMOP2(10) 6.21E-03(7.87E-03) 2.35E-01(1.79E-02)$†$ 4.08E-01(1.03E-01) 1.49E-01(2.64E-02)$†$ 5.66E-03(1.36E-03) 1.27E-03(7.24E-04)$‡$
DAS-CMOP2(11) 1.32E-03(6.25E-06) 5.40E-01(2.08E-01)$†$ 2.56E-01(8.50E-02) 6.98E-02(2.44E-01)$†$ 3.46E-03(1.17E-04) 6.89E-04(7.83E-04)$‡$
DAS-CMOP2(12) 4.54E-01(2.59E-01) 2.19E-01(2.43E-02)$‡$ 8.06E-02(1.79E-01) 1.32E-01(3.81E-02)$‡$ 5.93E-04(5.39E-04) 1.37E-03(2.57E-03)
DAS-CMOP2(13) 3.10E-01(2.33E-02) 3.01E-01(1.82E-02) 3.97E-03(8.47E-03) 1.18E-02(1.79E-02)$‡$ 6.22E-04(3.23E-04) 1.44E-03(1.41E-03)$‡$
DAS-CMOP2(14) 3.31E-01(1.70E-02) 3.25E-01(1.86E-02) 1.11E-02(1.86E-02) 3.30E-02(4.03E-02)$‡$ 5.18E-04(3.00E-04) 4.02E-04(3.26E-04)
DAS-CMOP2(15) 3.46E-01(1.16E-02) 3.43E-01(9.25E-03) 3.27E-03(8.68E-03) 1.83E-02(1.82E-02)$†$ 6.69E-04(4.55E-04 6.24E-04(7.77E-04)
DAS-CMOP2(16) 3.12E-01(1.36E-02) 3.09E-01(5.59E-03) 1.13E-02(2.08E-02) 2.88E-02(3.40E-02)$‡$ 4.47E-04(3.20E-04) 3.25E-04(2.39E-04)
DAS-CMOP3(1) 3.10E-02(5.25E-02) 3.37E-01(5.00E-02)$†$ 2.08E-01(7.85E-02) 4.54E-02(1.83E-01)$†$ 3.35E-03(2.82E-04) 1.28E-03(1.31E-03)$‡$
DAS-CMOP3(2) 1.57E-01(3.10E-02) 2.31E-01(1.47E-02)$†$ 2.55E-01(2.26E-01) 1.00E-01(1.51E-01)$†$ 2.99E-03(6.78E-04) 2.89E-03(2.42E-03)$‡$
DAS-CMOP3(3) 2.23E-02(5.23E-02) 2.84E-01(1.50E-02)$†$ 1.35E-01(8.22E-02) 2.48E-01(6.75E-02)$‡$ 3.72E-03(4.87E-04) 6.41E-04(4.40E-04)$‡$
DAS-CMOP3(4) 2.92E-01(8.95E-04) 2.94E-01(4.35E-03)$†$ 7.83E-02(4.14E-02) 4.84E-02(4.46E-02)$†$ 2.25E-03(4.79E-04) 1.71E-03(4.11E-03)$‡$
DAS-CMOP3(5) 9.97E-02(5.29E-02) 4.25E-01(1.06E-01)$†$ 2.51E-01(2.11E-01) 2.00E-01(4.07E-01)$†$ 4.27E-03(8.49E-04) NaN(NaN)
DAS-CMOP3(6) 1.67E-01(2.58E-02) 2.30E-01(1.17E-02)$†$ 3.11E-01(1.67E-01) 4.31E-01(2.30E-01)$‡$ 2.86E-03(5.53E-04) 2.93E-03(2.86E-03)$†$
DAS-CMOP3(7) 3.24E-02(6.16E-02) 2.67E-01(1.83E-02)$†$ 1.30E-01(6.73E-02) 4.31E-01(8.87E-02)$‡$ 3.60E-03(5.69E-04) 9.46E-04(8.74E-04)$‡$
DAS-CMOP3(8) 3.94E-01(1.39E-01) 3.42E-01(2.84E-02) 2.79E-03(4.02E-03) 1.03E-01(9.18E-02)$‡$ 7.94E-04(7.01E-04) 1.05E-03(5.11E-03)$†$
DAS-CMOP3(9) 1.67E-01(8.28E-02) 3.08E-01(4.50E-02)$†$ 2.29E-01(3.51E-01) 3.00E-01(4.66E-01) 1.35E-03(1.68E-03) NaN(NaN)
DAS-CMOP3(10) 1.70E-01(3.54E-02) 2.32E-01(2.10E-02)$†$ 1.85E-01(7.23E-02) 3.35E-01(2.46E-01) 2.81E-03(4.57E-04) 5.11E-03(3.37E-03)
DAS-CMOP3(11) 6.83E-02(9.70E-02) 2.42E-01(2.56E-02)$†$ 2.46E-02(5.21E-02) 2.85E-01(1.15E-01)$‡$ 3.54E-03(8.60E-04) 9.97E-04(9.11E-04)$‡$
DAS-CMOP3(12) 8.28E-01(2.38E-01) 7.95E-01(2.83E-01) 1.33E-01(3.46E-01) 4.70E-02(1.85E-01) 7.14E-05(2.29E-04) 2.47E-05(5.65E-05)
DAS-CMOP3(13) 3.45E-01(7.69E-02) 3.39E-01(9.36E-02) 4.55E-02(8.47E-02) 4.71E-02(7.48E-02) 1.62E-03(1.19E-03) NaN(NaN)
DAS-CMOP3(14) 4.48E-01(3.25E-02) 4.33E-01(4.47E-02) 3.64E-01(3.40E-01) 4.36E-02(2.89E-02) 1.41E-03(7.86E-04) NaN(NaN)
DAS-CMOP3(15) 5.37E-01(2.72E-01) 3.03E-01(3.75E-02)$‡$ 5.74E-02(1.83E-01) 3.43E-02(3.44E-02)$†$ 6.58E-04(7.88E-04) 1.97E-03(2.25E-03)$†$
DAS-CMOP3(16) 6.54E-01(1.69E-01) 4.73E-01(6.34E-02)$‡$ 5.33E-01(5.07E-01) 1.04E-01(3.04E-01)$†$ 3.08E-05(7.80E-05) NaN(NaN)
DAS-CMOP4(1) 5.00E-02(7.13E-02) 1.04E-03(3.40E-05)$‡$ 6.06E-01(4.92E-01) 9.22E-03(3.26E-02)$†$ 3.26E-03(3.91E-04) 1.56E-03(1.05E-04)$‡$
DAS-CMOP4(2) 1.81E-03(6.90E-04) 1.60E-03(9.90E-05) 1.72E-01(4.20E-02) 1.06E-01(1.55E-02)$‡$ 2.32E-03(7.85E-04) 1.73E-03(1.45E-04)$‡$
DAS-CMOP4(3) 8.94E-02(1.78E-01) 1.57E-01(9.91E-02)$†$ 6.77E-02(2.16E-01) 1.36E-01(3.45E-01) 2.37E-03(1.51E-03) 1.00E-02(1.03E-02)$†$
DAS-CMOP4(4) 6.90E-02(6.98E-02) 6.94E-03(2.92E-02)$‡$ 2.78E-01(7.71E-02) 9.08E-02(2.40E-02)$†$ 3.37E-03(5.60E-04) 1.28E-03(2.22E-04)$‡$
DAS-CMOP4(5) 9.66E-02(9.33E-02) 9.91E-04(1.78E-05)$‡$ 8.43E-01(3.22E-01) 7.67E-03(1.89E-02)$†$ 4.05E-03(3.51E-03) 1.14E-03(6.30E-05)$‡$
DAS-CMOP4(6) 1.69E-03(4.57E-04) 1.58E-03(6.20E-05) 6.28E-02(2.89E-02) 1.71E-01(3.13E-02)$‡$ 2.35E-03(5.04E-04) 1.76E-03(1.64E-04)$‡$
DAS-CMOP4(7) 1.13E-01(1.40E-01) 4.90E-01(2.37E-01)$†$ 7.07E-01(3.29E-01) 8.56E-01(3.29E-01)$‡$ 3.28E-03(1.80E-03) 7.73E-03(4.91E-03)$†$
DAS-CMOP4(8) 2.04E-01(2.70E-05) 1.11E-01(1.57E-01)$‡$ 1.12E-01(4.60E-02) 7.98E-02(6.18E-02) 2.95E-03(7.52E-04) 4.01E-03(6.26E-03)$†$
DAS-CMOP4(9) 1.22E-01(1.54E-01) 5.40E-04(3.18E-05)$‡$ 2.15E-01(3.70E-01) 6.11E-03(2.69E-02)$†$ 3.92E-03(3.17E-03) 7.60E-04(4.76E-05)$‡$
DAS-CMOP4(10) 1.48E-03(1.13E-04) 1.57E-03(6.01E-05)$†$ 8.73E-02(2.59E-02) 9.89E-02(1.66E-02)$‡$ 2.29E-03(2.43E-04) 1.72E-03(1.64E-04)$‡$
DAS-CMOP4(11) 2.47E-01(2.30E-01) 8.53E-01(3.27E-01)$†$ 7.46E-01(2.50E-01) 9.51E-02(1.94E-01)$†$ 4.04E-03(2.50E-03) 2.29E-02(9.07E-02)$†$
DAS-CMOP4(12) 2.33E-01(6.95E-05) 2.33E-01(1.48E-04) 9.59E-03(1.11E-02) 1.55E-01(4.84E-02)$‡$ 1.98E-03(1.40E-04) 2.25E-04(3.22E-05)$‡$
DAS-CMOP4(13) 1.44E-03(6.30E-05) 3.79E-03(7.67E-03)$†$ 7.25E-02(3.87E-02) 6.67E-03(5.32E-03)$†$ 2.69E-03(3.65E-04) 2.52E-03(1.03E-03)$‡$
DAS-CMOP4(14) 1.59E-03(9.75E-05) 2.22E-02(6.76E-02)$†$ 1.72E-01(8.13E-02) 6.67E-03(6.06E-03)$†$ 3.56E-03(2.01E-04) 1.63E-03(2.42E-03)$‡$
DAS-CMOP4(15) 2.24E-01(4.07E-05) 2.25E-01(1.23E-03)$†$ 1.12E-01(1.08E-01) 1.11E-03(3.20E-03)$†$ 2.07E-03(3.64E-04) 1.41E-03(2.07E-03)$‡$
DAS-CMOP4(16) 2.72E-01(7.92E-05) 3.02E-01(9.15E-02)$†$ 4.40E-02(3.17E-02) 4.00E-03(2.07E-02)$†$ 2.50E-03(2.13E-04) 6.99E-04(1.27E-03)$‡$
DAS-CMOP5(1) 1.30E-03(8.22E-07) 1.19E-03(7.00E-05)$‡$ 1.66E-01(3.59E-02) 3.67E-03(3.08E-03)$†$ 4.59E-03(3.39E-05) 2.19E-03(2.11E-04)$‡$
DAS-CMOP5(2) 1.95E-03(1.91E-04) 1.48E-03(2.12E-05)$‡$ 3.44E-01(9.58E-02) 9.67E-03(3.54E-03)$†$ 5.19E-03(7.01E-04) 2.28E-03(9.71E-05)$‡$
DAS-CMOP5(3) 1.29E-03(8.64E-07) 1.54E-03(3.93E-05)$†$ 1.70E-01(2.15E-02) 4.67E-03(2.85E-03)$†$ 3.41E-03(6.50E-05) 2.48E-03(2.13E-04)$‡$
DAS-CMOP5(4) 1.93E-03(2.19E-04) 1.08E-03(2.45E-05)$‡$ 3.29E-01(5.88E-02) 1.00E-02(5.81E-03)$†$ 5.35E-03(5.06E-04) 1.89E-03(1.21E-04)$‡$
DAS-CMOP5(5) 1.44E-03(1.42E-06) 1.05E-03(6.39E-05)$‡$ 8.65E-02(5.02E-02) 5.00E-03(3.25E-03)$†$ 4.92E-03(5.17E-05) 1.73E-03(1.83E-04)$‡$
DAS-CMOP5(6) 1.94E-03(3.12E-04) 1.48E-03(2.84E-05)$‡$ 1.70E-01(5.16E-02) 3.00E-03(1.60E-03)$†$ 4.96E-03(6.67E-04) 2.32E-03(1.28E-04)$‡$
DAS-CMOP5(7) 1.29E-03(1.25E-06) 1.52E-03(4.73E-05)$†$ 1.50E-01(2.67E-02) 5.11E-03(4.77E-03)$†$ 3.40E-03(7.56E-05) 2.40E-03(1.27E-04)$‡$
DAS-CMOP5(8) 2.03E-03(1.48E-04) 1.02E-03(2.15E-05)$‡$ 3.64E-01(7.98E-02) 5.86E-02(1.20E-02)$†$ 5.18E-03(3.79E-04) 1.50E-03(8.35E-05)$‡$
DAS-CMOP5(9) 1.22E-03(1.10E-06) 5.94E-04(4.78E-05)$‡$ 3.21E-01(9.32E-02) 2.89E-03(1.15E-03)$†$ 3.50E-03(3.51E-05) 1.10E-03(1.28E-04)$‡$
DAS-CMOP5(10) 1.92E-03(3.83E-04) 1.48E-03(3.41E-05)$‡$ 2.16E-01(3.17E-02) 0.00E+00(0.00E+00)$†$ 4.75E-03(6.92E-04) 2.26E-03(1.37E-04)$‡$
DAS-CMOP5(11) 1.29E-03(8.23E-07) 1.51E-03(4.05E-05)$†$ 1.23E-01(2.95E-02) 8.89E-03(5.20E-03)$†$ 3.41E-03(6.89E-05) 2.41E-03(1.42E-04)$‡$
DAS-CMOP5(12) 1.17E-01(3.15E-01) 1.39E-02(4.24E-04) 1.52E-01(7.85E-02) 7.84E-02(1.26E-02)$†$ 4.03E-03(1.40E-03) 1.05E-03(4.64E-05)$‡$
DAS-CMOP5(13) 1.76E-03(1.91E-04) 1.50E-03(2.98E-05)$‡$ 1.36E-01(6.73E-02) 1.67E-03(5.52E-03)$†$ 4.54E-03(7.31E-04) 2.37E-03(1.23E-04)$‡$
DAS-CMOP5(14) 1.87E-03(2.08E-04) 9.73E-04(2.67E-05)$‡$ 2.91E-01(1.42E-01) 3.78E-03(3.36E-03)$†$ 4.94E-03(3.61E-04) 1.45E-03(9.60E-05)$‡$
DAS-CMOP5(15) 3.02E-01(9.47E-02) 6.26E-02(9.70E-02)$‡$ 1.22E-01(1.16E-01) 2.22E-03(2.53E-03)$†$ 3.54E-03(8.95E-04) 1.50E-03(2.63E-04)$‡$
DAS-CMOP5(16) 3.32E-01(1.04E-01) 6.70E-02(8.60E-02)$‡$ 3.24E-02(7.80E-02) 3.78E-03(3.00E-03)$†$ 2.89E-03(1.01E-03) 8.97E-04(1.00E-04)$‡$

For DAS-CMOP3 with equality constraints, NSGA-II-CDP is better or significantly better than MOEA/D-CDP in $IGD$ values, and for DAS-CMOP3 with the remainder of the difficulty triplets, MOEA/D is significantly better than NSGA-II-CDP except for the case with the difficulty triplet (0.5, 0.5, 0.5). For DAS-CMOP4 with equality constraints, MOEA/D-CDP is significantly better than NSGA-II-CDP in $IGD$ values. For DAS-CMOP4 with diversity-hardness—that is, the difficulty triplets are (0.25, 0.0, 0.0), (0.5, 0.0, 0.0), and (0.75, 0.0, 0.0)—NSGA-II-CDP is significantly better than MOEA/D-CDP in $IGD$ values.

For DAS-CMOP5 with convergence-hardness—that is, the difficulty triplets are (0.0, 0.0, 0.25), (0.0, 0.0, 0.5), and (0.0, 0.0, 0.75)—MOEA/D-CDP is significantly better than NSGA-II-CDP in $IGD$ values. For DAS-CMOP5 with the rest of the difficulty triplets, NSGA-II-CDP is significantly better than MOEA/D-CDP in most cases in $IGD$ values.

In terms of $C$-metric values, NSGA-II-CDP and MOEA/D-CDP perform almost the same on DAS-CMOP1-4. For DAS-CMOP5, MOEA/D-CDP is significantly better than NSGA-II-CDP in $C$-metric values. In terms of spacing values, NSGA-II-CDP is better or significantly better than MOEA/D-CDP on DAS-CMOP1-5 in most cases.

Table 5 shows the statistics on $IGD$, $C$-metric and spacing values for MOEA/D-CDP and NSGA-II-CDP on DAS-CMOP6-9 with sixteen different difficulty triplets as shown in Table 3. We can see that for DAS-CMOP6 with convergence-hardness, MOEA/D-CDP is significantly better than NSGA-II-CDP in $IGD$ values. For DAS-CMOP6 with diversity-hardness, NSGA-II-CDP is better or significantly better than MOEA/D-CDP in $IGD$ values.

Table 5:
Means and standard deviations of $IGD$, $C$-metric, and spacing values obtained by MOEA/D-CDP and NSGA-II-CDP on DAS-CMOP6-9 with sixteen different difficulty triplets. Wilcoxon's rank sum test at 0.05 significance level is performed between MOEA/D-CDP and NSGA-II-CDP. $†$ and $‡$ denote that the performance of NSGA-II-CDP is significantly worse than or better than that of MOEA/D-CDP, respectively. DAS-CMOP6(i) means that DAS-CMOP6 has the $i$-th difficulty triplet in Table 3.
$IGD$$C$-metricSpacing
Test ProblemMOEA/D-CDPNSGA-II-CDPMOEA/D-CDPNSGA-II-CDPMOEA/D-CDPNSGA-II-CDP
DAS-CMOP6(1) 8.21E-02(1.29E-01) 2.66E-02(3.30E-02)$‡$ 2.14E-01(3.38E-01) 5.27E-02(7.57E-02) 3.23E-03(9.82E-04) 5.18E-04(1.28E-04)$‡$
DAS-CMOP6(2) 6.16E-02(7.09E-02) 1.54E-02(2.12E-02)$‡$ 9.03E-02(5.92E-02) 1.40E-01(4.94E-02)$‡$ 3.64E-03(1.29E-03) 1.80E-03(2.28E-03)$‡$
DAS-CMOP6(3) 7.76E-02(1.47E-01) 1.28E-01(1.23E-01) 4.41E-01(3.60E-01) 7.69E-01(3.45E-01)$‡$ 3.95E-03(9.34E-04) 1.51E-02(3.69E-02)
DAS-CMOP6(4) 1.72E-01(2.08E-01) 3.85E-02(4.75E-02)$‡$ 9.74E-02(1.12E-01) 1.90E-01(8.09E-02)$‡$ 3.97E-03(9.16E-03) 1.72E-03(3.03E-03)$‡$
DAS-CMOP6(5) 1.35E-01(8.55E-02) 5.86E-02(4.18E-02)$‡$ 3.01E-01(3.16E-01) 1.85E-01(3.04E-01) 4.01E-03(3.87E-03) 3.77E-02(4.53E-03)$‡$
DAS-CMOP6(6) 3.47E-02(4.22E-02) 1.37E-02(1.86E-02)$‡$ 2.00E-01(6.61E-02) 1.84E-01(3.84E-02) 3.37E-03(4.50E-04) 1.43E-03(5.40E-04)$‡$
DAS-CMOP6(7) 2.23E-01(3.76E-01) 5.31E-01(2.43E-01)$†$ 2.26E-01(3.42E-01) 1.64E-01(1.86E-01) 3.88E-03(1.67E-03) 1.07E-02(7.54E-03)$‡$
DAS-CMOP6(8) 6.50E-01(6.44E-01) 1.90E-01(2.90E-01)$‡$ 4.81E-01(4.73E-01) 6.08E-02(5.83E-02)$†$ 1.06E-03(1.10E-03) 3.10E-02(1.30E-02)$†$
DAS-CMOP6(9) 9.57E-02(1.50E-01) 7.74E-03(1.58E-02) 2.28E-01(3.23E-01) 3.60E-01(3.58E-01) 6.89E-03(3.74E-02) 1.39E-01(5.51E-02)$†$
DAS-CMOP6(10) 3.25E-02(4.72E-02) 3.45E-02(3.47E-02) 1.75E-01(5.19E-02) 8.73E-02(2.78E-02)$†$ 3.57E-03(6.32E-04) 1.96E-03(3.01E-03)$‡$
DAS-CMOP6(11) 5.43E-01(6.27E-01) 7.82E-01(3.34E-01) 5.67E-01(5.04E-01) 1.02E-01(1.14E-01)$†$ 3.61E-03(9.31E-04) 1.28E-02(1.38E-02)$†$
DAS-CMOP6(12) 7.81E-01(5.56E-01) 4.92E-01(5.50E-01) 1.45E-01(2.99E-01) 1.04E-01(1.02E-01) 4.83E-04(8.77E-04) NaN(NaN)
DAS-CMOP6(13) 7.07E-03(2.77E-02) 3.86E-02(4.60E-02)$†$ 1.28E-01(8.68E-02) 7.89E-03(1.54E-02)$†$ 4.11E-03(2.77E-04) 1.09E-03(1.70E-04)$‡$
DAS-CMOP6(14) 3.65E-02(1.85E-02) 9.77E-02(7.51E-02)$†$ 7.30E-01(1.79E-01) 1.64E-01(4.96E-02)$†$ 8.29E-04(4.75E-04) 3.12E-02(7.03E-03)$†$
DAS-CMOP6(15) 4.63E-01(6.07E-01) 3.22E-02(4.47E-02) 6.33E-01(4.90E-01) 7.67E-03(6.07E-03)$†$ 2.49E-03(1.99E-03) 2.00E-03(2.08E-03)
DAS-CMOP6(16) 5.21E-01(6.09E-01) 2.34E-01(2.46E-01) 5.59E-02(5.61E-02) 2.36E-03(3.01E-03)$†$ 6.09E-04(6.75E-04) 2.43E-02(1.31E-02)$†$
DAS-CMOP7(1) 3.27E-01(5.19E-01) 2.90E-02(5.96E-03)$‡$ 4.69E-02(1.71E-01) 1.73E-02(5.07E-02)$†$ 3.19E-02(4.81E-03) 2.56E-02(1.31E-03)$‡$
DAS-CMOP7(2) 5.10E-02(5.35E-02) 3.00E-02(6.90E-04)$‡$ 3.30E-02(2.70E-02) 8.22E-02(1.20E-02)$‡$ 3.23E-02(2.70E-03) 2.43E-02(1.64E-03)$‡$
DAS-CMOP7(3) 1.02E-01(1.60E-01) 3.63E-02(6.98E-03) 4.46E-02(8.24E-02) 1.92E-01(1.42E-01)$‡$ 2.81E-02(2.91E-03) 2.46E-02(1.25E-03)$‡$
DAS-CMOP7(4) 4.24E-01(2.09E-01) 2.63E-02(6.39E-04)$‡$ 7.86E-01(2.90E-01) 8.00E-02(1.36E-02)$†$ 3.66E-02(4.41E-03) 2.52E-02(9.84E-04)$‡$
DAS-CMOP7(5) 1.02E+00(7.46E-01) 2.56E-02(5.98E-03)$‡$ 7.20E-01(4.24E-01) 2.37E-02(1.62E-02)$†$ 2.84E-02(1.12E-02) 2.42E-02(1.44E-03)$‡$
DAS-CMOP7(6) 2.98E-02(7.19E-04) 2.97E-02(8.50E-04) 4.46E-02(1.91E-02) 7.47E-02(1.82E-02)$‡$ 3.09E-02(1.88E-03) 2.44E-02(1.32E-03)$‡$
DAS-CMOP7(7) 1.48E-01(2.59E-01) 4.36E-02(1.23E-02) 6.79E-01(4.26E-01) 5.28E-02(9.86E-02)$†$ 2.99E-02(4.08E-03) 2.34E-02(1.89E-03)$‡$
DAS-CMOP7(8) 1.02E-01(7.00E-02) 2.25E-02(7.88E-04)$‡$ 5.51E-01(3.50E-01) 1.01E-01(1.76E-02)$†$ 3.18E-02(4.07E-03) 2.33E-02(1.43E-03)$‡$
DAS-CMOP7(9) 7.47E-01(1.03E+00) 1.94E-02(3.37E-03)$‡$ 2.83E-01(4.14E-01) 4.44E-03(6.91E-03) 3.00E-02(1.39E-02) 2.15E-02(2.07E-03)$‡$
DAS-CMOP7(10) 3.01E-02(1.12E-03) 3.02E-02(7.46E-04) 2.40E-02(2.44E-02) 6.78E-02(1.39E-02)$‡$ 3.14E-02(2.32E-03) 2.46E-02(1.20E-03)$‡$
DAS-CMOP7(11) 1.19E-01(1.66E-01) 4.49E-02(1.65E-02) 1.71E-01(2.74E-01) 4.52E-01(3.32E-01)$‡$ 2.91E-02(1.66E-03) 2.17E-02(1.21E-03)$‡$
DAS-CMOP7(12) 2.61E-02(7.97E-03) 1.74E-02(1.66E-03)$‡$ 7.77E-02(3.20E-02) 1.08E-01(1.54E-02)$‡$ 2.73E-02(2.91E-03) 2.06E-02(1.92E-03)$‡$
DAS-CMOP7(13) 3.05E-02(2.88E-03) 4.09E-02(4.65E-02)$†$ 1.13E-02(1.96E-02) 2.22E-04(8.46E-04)$†$ 3.42E-02(4.44E-03) 2.43E-02(2.64E-03)$‡$
DAS-CMOP7(14) 2.43E-02(9.99E-04) 5.27E-02(7.61E-02) 1.45E-02(1.65E-02) 2.22E-04(8.46E-04)$†$ 3.30E-02(1.69E-03) 2.18E-02(4.17E-03)$‡$
DAS-CMOP7(15) 3.04E-02(1.21E-03) 3.12E-02(1.24E-02)$†$ 8.90E-02(5.28E-02) 0.00E+00(0.00E+00)$†$ 3.47E-02(2.24E-03) 2.46E-02(1.84E-03)$‡$
DAS-CMOP7(16) 2.41E-02(1.05E-03) 3.82E-02(5.57E-02)$†$ 9.54E-03(5.50E-03) 0.00E+00(0.00E+00)$†$ 3.26E-02(1.38E-03) 2.29E-02(3.12E-03)$‡$
DAS-CMOP8(1) 8.95E-02(9.95E-02) 3.58E-02(3.50E-03)$‡$ 1.15E-01(2.01E-01) 9.67E-03(9.28E-03) 4.43E-02(4.22E-03) 3.13E-02(1.44E-03)$‡$
DAS-CMOP8(2) 4.32E-02(5.06E-03) 3.90E-02(9.67E-04)$‡$ 1.29E-01(4.96E-02) 6.86E-02(1.75E-02)$†$ 4.58E-02(4.76E-03) 3.05E-02(1.42E-03)$‡$
DAS-CMOP8(3) 9.61E-02(1.13E-01) 4.44E-02(4.84E-03) 9.32E-02(1.19E-01) 3.31E-01(1.76E-01)$‡$ 4.10E-02(4.20E-03) 3.23E-02(1.80E-03)$‡$
DAS-CMOP8(4) 1.36E-01(8.71E-02) 3.37E-02(1.03E-03)$‡$ 9.41E-02(1.29E-01) 7.20E-02(1.43E-02) 4.79E-02(7.01E-03) 3.14E-02(1.31E-03)$‡$
DAS-CMOP8(5) 3.58E-01(2.45E-01) 3.03E-02(3.77E-03)$‡$ 3.40E-01(4.57E-01) 3.88E-01(1.59E-01) 4.45E-02(6.95E-03) 2.91E-02(2.03E-03)$‡$
DAS-CMOP8(6) 3.95E-02(1.53E-03) 3.94E-02(1.03E-03) 5.15E-02(5.52E-02) 7.42E-02(1.93E-02)$‡$ 4.59E-02(2.95E-03) 3.06E-02(1.59E-03)$‡$
DAS-CMOP8(7) 8.61E-02(7.46E-02) 5.10E-02(6.52E-03)$‡$ 8.19E-02(1.22E-01) 1.44E-01(1.52E-01)$‡$ 3.84E-02(3.16E-03) 3.08E-02(1.11E-03)$‡$
DAS-CMOP8(8) 6.18E-02(3.66E-02) 2.86E-02(1.15E-03)$‡$ 6.51E-02(6.02E-02) 8.81E-02(1.36E-02)$‡$ 4.42E-02(5.54E-03) 2.89E-02(1.55E-03)$‡$
DAS-CMOP8(9) 6.23E-01(7.87E-01) 2.29E-02(2.81E-03)$‡$ 3.48E-01(4.42E-01) 2.39E-01(9.66E-02) 2.85E-02(1.76E-02) 2.69E-02(2.56E-03)
DAS-CMOP8(10) 3.95E-02(1.61E-03) 3.90E-02(1.18E-03) 2.96E-02(1.40E-02) 6.34E-02(1.92E-02)$‡$ 4.61E-02(2.52E-03) 3.05E-02(1.78E-03)$‡$
DAS-CMOP8(11) 1.26E-01(1.29E-01) 5.48E-02(1.07E-02)$‡$ 9.37E-01(2.38E-01) 4.32E-01(1.91E-01)$†$ 3.68E-02(5.36E-03) 2.69E-02(1.68E-03)$‡$
DAS-CMOP8(12) 3.26E-02(7.91E-03) 2.05E-02(1.34E-03)$‡$ 1.84E-01(3.79E-02) 7.98E-02(2.09E-02)$†$ 4.32E-02(3.03E-03) 2.59E-02(1.91E-03)$‡$
DAS-CMOP8(13) 3.90E-02(1.32E-03) 5.69E-02(6.05E-02) 4.25E-02(5.14E-02) 2.89E-03(4.08E-03)$†$ 4.93E-02(3.15E-03) 2.99E-02(4.63E-03)$‡$
DAS-CMOP8(14) 3.42E-02(2.24E-03) 7.04E-02(1.12E-01)$†$ 1.28E-01(6.76E-02) 0.00E+00(0.00E+00)$†$ 4.72E-02(2.14E-03) 2.64E-02(7.02E-03)$‡$
DAS-CMOP8(15) 3.91E-02(1.99E-03) 6.38E-02(9.68E-02) 3.26E-02(3.10E-02) 3.78E-03(3.58E-03)$†$ 4.93E-02(4.92E-03) 3.10E-02(3.43E-03)$‡$
DAS-CMOP8(16) 3.34E-02(1.79E-03) 8.14E-02(1.24E-01)$†$ 6.53E-02(3.61E-02) 0.00E+00(0.00E+00)$†$ 4.72E-02(2.74E-03) 2.60E-02(6.48E-03)$‡$
DAS-CMOP9(1) 1.55E-01(2.68E-01) 4.60E-01(8.73E-02)$†$ 6.59E-02(4.45E-02) 1.39E-01(8.20E-02)$‡$ 3.57E-02(1.61E-02) 8.16E-03(5.20E-03)$‡$
DAS-CMOP9(2) 4.92E-02(2.62E-03) 1.63E-01(6.57E-02)$†$ 2.67E-01(1.59E-01) 3.81E-02(9.37E-03)$†$ 6.28E-02(3.11E-03) 2.38E-02(5.98E-03)$‡$
DAS-CMOP9(3) 3.87E-02(2.39E-04) 4.38E-01(8.90E-02)$†$ 1.06E-01(2.48E-02) 1.01E-01(7.73E-02) 3.66E-02(5.93E-04) 8.68E-03(5.48E-03)$‡$
DAS-CMOP9(4) 4.85E-02(2.70E-03) 2.22E-01(9.75E-02)$†$ 2.91E-01(8.59E-02) 4.58E-02(1.34E-02)$†$ 6.09E-02(3.30E-03) 2.03E-02(6.77E-03)$‡$
DAS-CMOP9(5) 3.31E-02(5.93E-04) 4.54E-01(1.09E-01)$†$ 1.23E-01(3.24E-02) 1.03E-01(6.75E-02) 4.28E-02(7.89E-04) 8.56E-03(6.05E-03)$‡$
DAS-CMOP9(6) 4.84E-02(2.33E-03) 1.91E-01(7.97E-02)$†$ 2.29E-01(1.78E-01) 4.48E-02(1.65E-02)$†$ 6.24E-02(2.47E-03) 2.09E-02(5.74E-03)$‡$
DAS-CMOP9(7) 4.06E-02(3.18E-04) 4.29E-01(8.42E-02)$†$ 9.92E-02(2.21E-02) 3.42E-02(4.92E-02)$†$ 3.58E-02(6.55E-04) 9.52E-03(5.45E-03)$‡$
DAS-CMOP9(8) 7.67E-02(1.03E-01) 2.27E-01(7.77E-02)$†$ 4.36E-02(2.59E-02) 5.32E-02(1.74E-02) 5.77E-02(1.41E-02) 1.78E-02(5.99E-03)$‡$
DAS-CMOP9(9) 6.36E-02(1.44E-01) 4.29E-01(1.44E-01)$†$ 1.03E-01(5.31E-02) 3.40E-02(6.25E-02)$†$ 3.81E-02(9.82E-03) 8.71E-03(8.06E-03)$‡$
DAS-CMOP9(10) 5.04E-02(3.16E-03) 1.66E-01(6.58E-02)$†$ 3.18E-01(1.09E-01) 4.97E-02(1.26E-02)$†$ 6.33E-02(2.44E-03) 2.34E-02(6.24E-03)$‡$
DAS-CMOP9(11) 4.18E-02(4.87E-04) 4.63E-01(6.67E-02)$†$ 1.19E-01(2.54E-02) 1.93E-01(1.09E-01)$†$ 3.33E-02(8.43E-04) 8.24E-03(3.98E-03)$‡$
DAS-CMOP9(12) 2.66E-01(1.76E-01) 2.68E-01(9.44E-02) 8.16E-03(1.11E-02) 7.88E-02(2.98E-02)$‡$ 3.08E-02(2.42E-02) 1.42E-02(7.94E-03)$‡$
DAS-CMOP9(13) 4.65E-01(6.30E-02) 4.67E-01(4.86E-02) 9.10E-02(1.87E-01) 2.11E-03(5.14E-03) 6.96E-03(3.67E-03) 6.22E-03(2.95E-03)
DAS-CMOP9(14) 5.98E-01(7.62E-02) 5.34E-01(5.24E-02)$‡$ 1.86E-03(5.74E-03) 1.78E-03(4.93E-03) 2.16E-03(8.30E-04) 3.58E-03(2.86E-03)$†$
DAS-CMOP9(15) 4.77E-01(7.26E-02) 4.75E-01(4.61E-02) 2.32E-01(1.89E-01) 2.22E-04(8.46E-04)$†$ 6.13E-03(3.27E-03) 6.36E-03(2.93E-03)
DAS-CMOP9(16) 6.05E-01(8.06E-02) 5.28E-01(4.95E-02)$‡$ 0.00E+00(0.00E+00) 3.33E-03(6.00E-03)$‡$ 2.27E-03(1.53E-03) 3.79E-03(2.06E-03)$‡$
$IGD$$C$-metricSpacing
Test ProblemMOEA/D-CDPNSGA-II-CDPMOEA/D-CDPNSGA-II-CDPMOEA/D-CDPNSGA-II-CDP
DAS-CMOP6(1) 8.21E-02(1.29E-01) 2.66E-02(3.30E-02)$‡$ 2.14E-01(3.38E-01) 5.27E-02(7.57E-02) 3.23E-03(9.82E-04) 5.18E-04(1.28E-04)$‡$
DAS-CMOP6(2) 6.16E-02(7.09E-02) 1.54E-02(2.12E-02)$‡$ 9.03E-02(5.92E-02) 1.40E-01(4.94E-02)$‡$ 3.64E-03(1.29E-03) 1.80E-03(2.28E-03)$‡$
DAS-CMOP6(3) 7.76E-02(1.47E-01) 1.28E-01(1.23E-01) 4.41E-01(3.60E-01) 7.69E-01(3.45E-01)$‡$ 3.95E-03(9.34E-04) 1.51E-02(3.69E-02)
DAS-CMOP6(4) 1.72E-01(2.08E-01) 3.85E-02(4.75E-02)$‡$ 9.74E-02(1.12E-01) 1.90E-01(8.09E-02)$‡$ 3.97E-03(9.16E-03) 1.72E-03(3.03E-03)$‡$
DAS-CMOP6(5) 1.35E-01(8.55E-02) 5.86E-02(4.18E-02)$‡$ 3.01E-01(3.16E-01) 1.85E-01(3.04E-01) 4.01E-03(3.87E-03) 3.77E-02(4.53E-03)$‡$
DAS-CMOP6(6) 3.47E-02(4.22E-02) 1.37E-02(1.86E-02)$‡$ 2.00E-01(6.61E-02) 1.84E-01(3.84E-02) 3.37E-03(4.50E-04) 1.43E-03(5.40E-04)$‡$
DAS-CMOP6(7) 2.23E-01(3.76E-01) 5.31E-01(2.43E-01)$†$ 2.26E-01(3.42E-01) 1.64E-01(1.86E-01) 3.88E-03(1.67E-03) 1.07E-02(7.54E-03)$‡$
DAS-CMOP6(8) 6.50E-01(6.44E-01) 1.90E-01(2.90E-01)$‡$ 4.81E-01(4.73E-01) 6.08E-02(5.83E-02)$†$ 1.06E-03(1.10E-03) 3.10E-02(1.30E-02)$†$
DAS-CMOP6(9) 9.57E-02(1.50E-01) 7.74E-03(1.58E-02) 2.28E-01(3.23E-01) 3.60E-01(3.58E-01) 6.89E-03(3.74E-02) 1.39E-01(5.51E-02)$†$
DAS-CMOP6(10) 3.25E-02(4.72E-02) 3.45E-02(3.47E-02) 1.75E-01(5.19E-02) 8.73E-02(2.78E-02)$†$ 3.57E-03(6.32E-04) 1.96E-03(3.01E-03)$‡$
DAS-CMOP6(11) 5.43E-01(6.27E-01) 7.82E-01(3.34E-01) 5.67E-01(5.04E-01) 1.02E-01(1.14E-01)$†$ 3.61E-03(9.31E-04) 1.28E-02(1.38E-02)$†$
DAS-CMOP6(12) 7.81E-01(5.56E-01) 4.92E-01(5.50E-01) 1.45E-01(2.99E-01) 1.04E-01(1.02E-01) 4.83E-04(8.77E-04) NaN(NaN)
DAS-CMOP6(13) 7.07E-03(2.77E-02) 3.86E-02(4.60E-02)$†$ 1.28E-01(8.68E-02) 7.89E-03(1.54E-02)$†$ 4.11E-03(2.77E-04) 1.09E-03(1.70E-04)$‡$
DAS-CMOP6(14) 3.65E-02(1.85E-02) 9.77E-02(7.51E-02)$†$ 7.30E-01(1.79E-01) 1.64E-01(4.96E-02)$†$ 8.29E-04(4.75E-04) 3.12E-02(7.03E-03)$†$
DAS-CMOP6(15) 4.63E-01(6.07E-01) 3.22E-02(4.47E-02) 6.33E-01(4.90E-01) 7.67E-03(6.07E-03)$†$ 2.49E-03(1.99E-03) 2.00E-03(2.08E-03)
DAS-CMOP6(16) 5.21E-01(6.09E-01) 2.34E-01(2.46E-01) 5.59E-02(5.61E-02) 2.36E-03(3.01E-03)$†$ 6.09E-04(6.75E-04) 2.43E-02(1.31E-02)$†$
DAS-CMOP7(1) 3.27E-01(5.19E-01) 2.90E-02(5.96E-03)$‡$ 4.69E-02(1.71E-01) 1.73E-02(5.07E-02)$†$ 3.19E-02(4.81E-03) 2.56E-02(1.31E-03)$‡$
DAS-CMOP7(2) 5.10E-02(5.35E-02) 3.00E-02(6.90E-04)$‡$ 3.30E-02(2.70E-02) 8.22E-02(1.20E-02)$‡$ 3.23E-02(2.70E-03) 2.43E-02(1.64E-03)$‡$
DAS-CMOP7(3) 1.02E-01(1.60E-01) 3.63E-02(6.98E-03) 4.46E-02(8.24E-02) 1.92E-01(1.42E-01)$‡$ 2.81E-02(2.91E-03) 2.46E-02(1.25E-03)$‡$
DAS-CMOP7(4) 4.24E-01(2.09E-01) 2.63E-02(6.39E-04)$‡$ 7.86E-01(2.90E-01) 8.00E-02(1.36E-02)$†$ 3.66E-02(4.41E-03) 2.52E-02(9.84E-04)$‡$
DAS-CMOP7(5) 1.02E+00(7.46E-01) 2.56E-02(5.98E-03)$‡$ 7.20E-01(4.24E-01) 2.37E-02(1.62E-02)$†$ 2.84E-02(1.12E-02) 2.42E-02(1.44E-03)$‡$
DAS-CMOP7(6) 2.98E-02(7.19E-04) 2.97E-02(8.50E-04) 4.46E-02(1.91E-02) 7.47E-02(1.82E-02)$‡$ 3.09E-02(1.88E-03) 2.44E-02(1.32E-03)$‡$
DAS-CMOP7(7) 1.48E-01(2.59E-01) 4.36E-02(1.23E-02) 6.79E-01(4.26E-01) 5.28E-02(9.86E-02)$†$ 2.99E-02(4.08E-03) 2.34E-02(1.89E-03)$‡$
DAS-CMOP7(8) 1.02E-01(7.00E-02) 2.25E-02(7.88E-04)$‡$ 5.51E-01(3.50E-01) 1.01E-01(1.76E-02)$†$ 3.18E-02(4.07E-03) 2.33E-02(1.43E-03)$‡$
DAS-CMOP7(9) 7.47E-01(1.03E+00) 1.94E-02(3.37E-03)$‡$ 2.83E-01(4.14E-01) 4.44E-03(6.91E-03) 3.00E-02(1.39E-02) 2.15E-02(2.07E-03)$‡$
DAS-CMOP7(10) 3.01E-02(1.12E-03) 3.02E-02(7.46E-04) 2.40E-02(2.44E-02) 6.78E-02(1.39E-02)$‡$ 3.14E-02(2.32E-03) 2.46E-02(1.20E-03)$‡$
DAS-CMOP7(11) 1.19E-01(1.66E-01) 4.49E-02(1.65E-02) 1.71E-01(2.74E-01) 4.52E-01(3.32E-01)$‡$ 2.91E-02(1.66E-03) 2.17E-02(1.21E-03)$‡$
DAS-CMOP7(12) 2.61E-02(7.97E-03) 1.74E-02(1.66E-03)$‡$ 7.77E-02(3.20E-02) 1.08E-01(1.54E-02)$‡$ 2.73E-02(2.91E-03) 2.06E-02(1.92E-03)$‡$
DAS-CMOP7(13) 3.05E-02(2.88E-03) 4.09E-02(4.65E-02)$†$ 1.13E-02(1.96E-02) 2.22E-04(8.46E-04)$†$ 3.42E-02(4.44E-03) 2.43E-02(2.64E-03)$‡$
DAS-CMOP7(14) 2.43E-02(9.99E-04) 5.27E-02(7.61E-02) 1.45E-02(1.65E-02) 2.22E-04(8.46E-04)$†$ 3.30E-02(1.69E-03) 2.18E-02(4.17E-03)$‡$
DAS-CMOP7(15) 3.04E-02(1.21E-03) 3.12E-02(1.24E-02)$†$ 8.90E-02(5.28E-02) 0.00E+00(0.00E+00)$†$ 3.47E-02(2.24E-03) 2.46E-02(1.84E-03)$‡$
DAS-CMOP7(16) 2.41E-02(1.05E-03) 3.82E-02(5.57E-02)$†$ 9.54E-03(5.50E-03) 0.00E+00(0.00E+00)$†$ 3.26E-02(1.38E-03) 2.29E-02(3.12E-03)$‡$
DAS-CMOP8(1) 8.95E-02(9.95E-02) 3.58E-02(3.50E-03)$‡$ 1.15E-01(2.01E-01) 9.67E-03(9.28E-03) 4.43E-02(4.22E-03) 3.13E-02(1.44E-03)$‡$
DAS-CMOP8(2) 4.32E-02(5.06E-03) 3.90E-02(9.67E-04)$‡$ 1.29E-01(4.96E-02) 6.86E-02(1.75E-02)$†$ 4.58E-02(4.76E-03) 3.05E-02(1.42E-03)$‡$
DAS-CMOP8(3) 9.61E-02(1.13E-01) 4.44E-02(4.84E-03) 9.32E-02(1.19E-01) 3.31E-01(1.76E-01)$‡$ 4.10E-02(4.20E-03) 3.23E-02(1.80E-03)$‡$
DAS-CMOP8(4) 1.36E-01(8.71E-02) 3.37E-02(1.03E-03)$‡$ 9.41E-02(1.29E-01) 7.20E-02(1.43E-02) 4.79E-02(7.01E-03) 3.14E-02(1.31E-03)$‡$
DAS-CMOP8(5) 3.58E-01(2.45E-01) 3.03E-02(3.77E-03)$‡$ 3.40E-01(4.57E-01) 3.88E-01(1.59E-01) 4.45E-02(6.95E-03) 2.91E-02(2.03E-03)$‡$
DAS-CMOP8(6) 3.95E-02(1.53E-03) 3.94E-02(1.03E-03) 5.15E-02(5.52E-02) 7.42E-02(1.93E-02)$‡$ 4.59E-02(2.95E-03) 3.06E-02(1.59E-03)$‡$
DAS-CMOP8(7) 8.61E-02(7.46E-02) 5.10E-02(6.52E-03)$‡$ 8.19E-02(1.22E-01) 1.44E-01(1.52E-01)$‡$ 3.84E-02(3.16E-03) 3.08E-02(1.11E-03)$‡$
DAS-CMOP8(8) 6.18E-02(3.66E-02) 2.86E-02(1.15E-03)$‡$ 6.51E-02(6.02E-02) 8.81E-02(1.36E-02)$‡$ 4.42E-02(5.54E-03) 2.89E-02(1.55E-03)$‡$
DAS-CMOP8(9) 6.23E-01(7.87E-01) 2.29E-02(2.81E-03)$‡$ 3.48E-01(4.42E-01) 2.39E-01(9.66E-02) 2.85E-02(1.76E-02) 2.69E-02(2.56E-03)
DAS-CMOP8(10) 3.95E-02(1.61E-03) 3.90E-02(1.18E-03) 2.96E-02(1.40E-02) 6.34E-02(1.92E-02)$‡$ 4.61E-02(2.52E-03) 3.05E-02(1.78E-03)$‡$
DAS-CMOP8(11) 1.26E-01(1.29E-01) 5.48E-02(1.07E-02)$‡$ 9.37E-01(2.38E-01) 4.32E-01(1.91E-01)$†$ 3.68E-02(5.36E-03) 2.69E-02(1.68E-03)$‡$
DAS-CMOP8(12) 3.26E-02(7.91E-03) 2.05E-02(1.34E-03)$‡$ 1.84E-01(3.79E-02) 7.98E-02(2.09E-02)$†$ 4.32E-02(3.03E-03) 2.59E-02(1.91E-03)$‡$
DAS-CMOP8(13) 3.90E-02(1.32E-03) 5.69E-02(6.05E-02) 4.25E-02(5.14E-02) 2.89E-03(4.08E-03)$†$ 4.93E-02(3.15E-03) 2.99E-02(4.63E-03)$‡$
DAS-CMOP8(14) 3.42E-02(2.24E-03) 7.04E-02(1.12E-01)$†$ 1.28E-01(6.76E-02) 0.00E+00(0.00E+00)$†$ 4.72E-02(2.14E-03) 2.64E-02(7.02E-03)$‡$
DAS-CMOP8(15) 3.91E-02(1.99E-03) 6.38E-02(9.68E-02) 3.26E-02(3.10E-02) 3.78E-03(3.58E-03)$†$ 4.93E-02(4.92E-03) 3.10E-02(3.43E-03)$‡$
DAS-CMOP8(16) 3.34E-02(1.79E-03) 8.14E-02(1.24E-01)$†$ 6.53E-02(3.61E-02) 0.00E+00(0.00E+00)$†$ 4.72E-02(2.74E-03) 2.60E-02(6.48E-03)$‡$
DAS-CMOP9(1) 1.55E-01(2.68E-01) 4.60E-01(8.73E-02)$†$ 6.59E-02(4.45E-02) 1.39E-01(8.20E-02)$‡$ 3.57E-02(1.61E-02) 8.16E-03(5.20E-03)$‡$
DAS-CMOP9(2) 4.92E-02(2.62E-03) 1.63E-01(6.57E-02)$†$ 2.67E-01(1.59E-01) 3.81E-02(9.37E-03)$†$ 6.28E-02(3.11E-03) 2.38E-02(5.98E-03)$‡$
DAS-CMOP9(3) 3.87E-02(2.39E-04) 4.38E-01(8.90E-02)$†$ 1.06E-01(2.48E-02) 1.01E-01(7.73E-02) 3.66E-02(5.93E-04) 8.68E-03(5.48E-03)$‡$
DAS-CMOP9(4) 4.85E-02(2.70E-03) 2.22E-01(9.75E-02)$†$ 2.91E-01(8.59E-02) 4.58E-02(1.34E-02)$†$ 6.09E-02(3.30E-03) 2.03E-02(6.77E-03)$‡$
DAS-CMOP9(5) 3.31E-02(5.93E-04) 4.54E-01(1.09E-01)$†$ 1.23E-01(3.24E-02) 1.03E-01(6.75E-02) 4.28E-02(7.89E-04) 8.56E-03(6.05E-03)$‡$
DAS-CMOP9(6) 4.84E-02(2.33E-03) 1.91E-01(7.97E-02)$†$ 2.29E-01(1.78E-01) 4.48E-02(1.65E-02)$†$ 6.24E-02(2.47E-03) 2.09E-02(5.74E-03)$‡$
DAS-CMOP9(7) 4.06E-02(3.18E-04) 4.29E-01(8.42E-02)$†$ 9.92E-02(2.21E-02) 3.42E-02(4.92E-02)$†$ 3.58E-02(6.55E-04) 9.52E-03(5.45E-03)$‡$
DAS-CMOP9(8) 7.67E-02(1.03E-01) 2.27E-01(7.77E-02)$†$ 4.36E-02(2.59E-02) 5.32E-02(1.74E-02) 5.77E-02(1.41E-02) 1.78E-02(5.99E-03)$‡$
DAS-CMOP9(9) 6.36E-02(1.44E-01) 4.29E-01(1.44E-01)$†$ 1.03E-01(5.31E-02) 3.40E-02(6.25E-02)$†$ 3.81E-02(9.82E-03) 8.71E-03(8.06E-03)$‡$
DAS-CMOP9(10) 5.04E-02(3.16E-03) 1.66E-01(6.58E-02)$†$ 3.18E-01(1.09E-01) 4.97E-02(1.26E-02)$†$ 6.33E-02(2.44E-03) 2.34E-02(6.24E-03)$‡$
DAS-CMOP9(11) 4.18E-02(4.87E-04) 4.63E-01(6.67E-02)$†$ 1.19E-01(2.54E-02) 1.93E-01(1.09E-01)$†$ 3.33E-02(8.43E-04) 8.24E-03(3.98E-03)$‡$
DAS-CMOP9(12) 2.66E-01(1.76E-01) 2.68E-01(9.44E-02) 8.16E-03(1.11E-02) 7.88E-02(2.98E-02)$‡$ 3.08E-02(2.42E-02) 1.42E-02(7.94E-03)$‡$
DAS-CMOP9(13) 4.65E-01(6.30E-02) 4.67E-01(4.86E-02) 9.10E-02(1.87E-01) 2.11E-03(5.14E-03) 6.96E-03(3.67E-03) 6.22E-03(2.95E-03)
DAS-CMOP9(14) 5.98E-01(7.62E-02) 5.34E-01(5.24E-02)$‡$ 1.86E-03(5.74E-03) 1.78E-03(4.93E-03) 2.16E-03(8.30E-04) 3.58E-03(2.86E-03)$†$
DAS-CMOP9(15) 4.77E-01(7.26E-02) 4.75E-01(4.61E-02) 2.32E-01(1.89E-01) 2.22E-04(8.46E-04)$†$ 6.13E-03(3.27E-03) 6.36E-03(2.93E-03)
DAS-CMOP9(16) 6.05E-01(8.06E-02) 5.28E-01(4.95E-02)$‡$ 0.00E+00(0.00E+00) 3.33E-03(6.00E-03)$‡$ 2.27E-03(1.53E-03) 3.79E-03(2.06E-03)$‡$

For DAS-CMOP7 and DAS-CMOP8 with equality constraints—that is, the difficulty triplets are (0.0, 1.0, 0.0), (0.5, 1.0, 0.0), (0.0, 1.0, 0.5), and (0.5, 1.0, 0.5)—MOEA/D-CDP is better or significantly better than NSGA-II-CDP in $IGD$ values. For DAS-CMOP7 and DAS-CMOP8 with the rest of difficulty triplets, NSGA-II-CDP is significantly better than MOEA/D-CDP in most cases. For DAS-CMOP9 with equality constraints, NSGA-II-CDP is better or significantly better than MOEA/D-CDP except for the case with the difficult triplet (0.0, 1.0, 0.0) in $IGD$ values. For DAS-CMOP9 with the other difficulty triplets, MOEA/D-CDP is better or significantly better than MOEA/D-CDP in $IGD$ values.

In terms of $C$-metric values, MOEA/D-CDP is better or significantly better than NSGA-II-CDP on DAS-CMOP6-9 in most cases. In terms of spacing values, NSGA-II-CDP is better or significantly better than MOEA/D-CDP on DAS-CMOP7-9 in most cases.

Figure 7 shows the feasible and nondominated solutions with the median $IGD$ values in 30 independent runs made using MOEA/D-CDP and NSGA-II-CDP on DAS-CMOP1-3 and DAS-CMOP6 with different difficulty triplets. We can see that MOEA/D-CDP is more suitable for solving convergence-hard DAS-CMOPs, whereas NSGA-II-CDP is more suitable for solving DAS-CMOPs with simultaneous diversity-, feasibility-, and convergence-hardness.

Figure 7:

The feasible and nondominated solutions with the median $IGD$ values in 30 independent runs by using MOEA/D-CDP and NSGA-II-CDP on DAS-CMOP1-3 and DAS-CMOP6 with different difficulty triplets. (a)–(f) show the populations achieved by MOEA/D-CDP and NSGA-II-CDP on DAS-CMOPs with convergence-hardness. (g)–(l) show the populations achieved by MOEA/D-CDP and NSGA-II-CDP on DAS-CMOPs with simultaneous diversity-, feasibility-, and convergence-hardness.

Figure 7:

The feasible and nondominated solutions with the median $IGD$ values in 30 independent runs by using MOEA/D-CDP and NSGA-II-CDP on DAS-CMOP1-3 and DAS-CMOP6 with different difficulty triplets. (a)–(f) show the populations achieved by MOEA/D-CDP and NSGA-II-CDP on DAS-CMOPs with convergence-hardness. (g)–(l) show the populations achieved by MOEA/D-CDP and NSGA-II-CDP on DAS-CMOPs with simultaneous diversity-, feasibility-, and convergence-hardness.

### 7.5  Performance Comparisons on DAS-CMaOPs

Table 6 presents the statistics on $HV$ values for C-MOEA/DD and C-NSGA-III on 5-, 8-, and 10-objective DAS-CMaOP1-6 with the first twelve difficulty triplets in Table 3. We can see that for 5- and 8-objective DAS-CMaOP1 with feasibility-hardness, C-NSGA-III is significantly better than C-MOEA/DD. For 5-objective DAS-CMaOP1 with diversity-hardness, C-MOEA/DD is significantly better than C-NSGA-III except for the case with the difficulty triplet $(0.25,0.0,0.0)$. For 8-objective DAS-CMaOP1 with diversity-hardness, C-MOEA/DD is significantly better than C-NSGA-III. For 10-objective DAS-CMaOP1, C-NSGA-III is significantly better than C-MOEA/DD in most cases.

Table 6:
Means and standard deviations of $HV$ values obtained by C-MOEA/DD and C-NSGA-III on DAS-CMaOP1-6 with 5, 8, and 10 objectives. Wilcoxon's rank sum test at 0.05 significance level is performed between C-MOEA/DD and C-NSGA-III. $†$ and $‡$ denote that the performance of C-NSGA-III is significantly worse than or better than that of C-MOEA/DD, respectively. DAS-CMaOP1(i) means that DAS-CMaOP1 has the $i$-th difficulty triplet in Table 3.
m $=$ 5m $=$ 8m $=$ 10
Test ProblemC-MOEA/DDC-NSGA-IIIC-MOEA/DDC-NSGA-IIIC-MOEA/DDC-NSGA-III
DAS-CMaOP1(1) 6.62E-01(3.34E-02) 6.71E-01(3.23E-02) 7.95E-01(2.54E-02) 7.63E-01(7.21E-02)$†$ 7.70E-01(4.77E-02) 8.27E-01(9.18E-03)$‡$
DAS-CMaOP1(2) 5.15E-01(7.02E-02) 5.81E-01(1.29E-02)$‡$ 4.78E-01(6.98E-02) 5.27E-01(2.19E-02)$‡$ 4.94E-01(7.05E-03) 5.00E-01(1.11E-02)$‡$
DAS-CMaOP1(3) 7.55E-01(2.56E-02) 7.67E-01(5.78E-02) 8.47E-01(1.24E-02) 8.45E-01(2.54E-02) 8.48E-01(1.72E-02) 8.63E-01(7.47E-03)$‡$
DAS-CMaOP1(4) 2.48E-01(1.35E-02) 4.16E-01(4.22E-02)$‡$ 2.83E-01(2.81E-02) 4.60E-01(1.95E-02)$‡$ 2.95E-01(2.19E-02) 4.85E-01(4.27E-03)$‡$
DAS-CMaOP1(5) 6.32E-01(3.20E-02) 6.04E-01(3.65E-02)$†$ 7.68E-01(3.94E-02) 7.09E-01(1.10E-01)$†$ 7.03E-01(6.09E-02) 8.22E-01(1.19E-02)$‡$
DAS-CMaOP1(6) 4.79E-01(5.89E-02) 5.74E-01(1.55E-02)$‡$ 3.39E-01(4.90E-02) 5.29E-01(2.15E-02)$‡$ 4.22E-01(5.90E-02) 4.95E-01(1.43E-02)$‡$
DAS-CMaOP1(7) 7.60E-01(2.18E-02) 7.53E-01(6.32E-02) 8.44E-01(2.29E-02) 8.38E-01(1.14E-02) 6.91E-01(7.33E-02) 8.65E-01(8.12E-03)$‡$
DAS-CMaOP1(8) 2.50E-01(1.52E-02) 3.36E-01(3.80E-02)$‡$ 2.63E-01(1.31E-02) 4.32E-01(6.31E-03)$‡$ 2.54E-01(1.39E-02) 3.97E-01(1.88E-03)$‡$
DAS-CMaOP1(9) 5.88E-01(3.49E-02) 5.41E-01(5.18E-02)$†$ 7.25E-01(7.12E-02) 6.59E-01(1.49E-01) 5.89E-01(8.45E-02) 8.00E-01(8.06E-02)$‡$
DAS-CMaOP1(10) 4.70E-01(5.73E-02) 5.68E-01(2.23E-02)$‡$ 3.52E-01(5.77E-02) 4.99E-01(2.24E-02)$‡$ 4.19E-01(4.25E-02) 4.89E-01(6.33E-03)$‡$
DAS-CMaOP1(11) 1.72E-01(1.81E-03) 1.84E-01(1.07E-03)$‡$ 1.06E-01(3.01E-03) 1.22E-01(1.18E-03)$‡$ 9.27E-02(1.19E-03) 9.10E-02(7.12E-04)$†$
DAS-CMaOP1(12) 2.41E-01(1.56E-02) 1.66E-01(5.86E-03)$†$ 2.48E-01(1.11E-02) 1.36E-01(4.71E-06)$†$ 2.44E-01(1.32E-02) 2.00E-01(4.25E-05)$†$
DAS-CMaOP2(1) 8.09E-01(3.54E-03) 8.10E-01(2.93E-03) 8.06E-01(5.53E-03) 8.09E-01(1.12E-03)$‡$ 8.07E-01(1.04E-03) 8.07E-01(1.28E-03)
DAS-CMaOP2(2) 6.04E-01(4.25E-02) 6.20E-01(2.64E-02)$‡$ 5.61E-01(1.67E-03) 5.66E-01(6.64E-05)$‡$ 5.37E-01(7.37E-04) 5.38E-01(6.51E-05)$‡$
DAS-CMaOP2(3) 9.61E-01(5.22E-02) 9.83E-01(3.93E-02)$‡$ 9.68E-01(3.40E-03) 9.97E-01(8.16E-04)$‡$ 9.65E-01(3.03E-03) 9.97E-01(8.92E-04)$‡$
DAS-CMaOP2(4) 5.06E-01(1.08E-03) 5.07E-01(6.87E-04)$‡$ 4.56E-01(7.10E-05) 4.56E-01(6.66E-04) 4.33E-01(4.98E-05) 4.33E-01(4.88E-04)
DAS-CMaOP2(5) 7.99E-01(4.74E-03) 8.03E-01(1.67E-03)$‡$ 8.01E-01(2.16E-03) 8.01E-01(2.38E-03) 8.01E-01(9.29E-04) 8.01E-01(1.43E-03)
DAS-CMaOP2(6) 6.15E-01(2.60E-02) 6.09E-01(4.32E-02)$†$ 5.60E-01(1.85E-03) 5.66E-01(7.16E-04)$‡$ 5.37E-01(6.81E-04) 5.38E-01(1.22E-04)$‡$
DAS-CMaOP2(7) 9.35E-01(7.55E-02) 8.94E-01(9.02E-02) 9.69E-01(3.97E-03) 9.96E-01(1.01E-03)$‡$ 9.63E-01(3.99E-03) 9.96E-01(7.62E-04)$‡$
DAS-CMaOP2(8) 4.98E-01(1.60E-03) 4.19E-01(7.53E-02)$†$ 3.95E-01(5.72E-02) 2.95E-01(9.37E-02)$‡$ 3.39E-01(4.82E-02) 2.75E-01(8.26E-02)$†$
DAS-CMaOP2(9) 7.82E-01(7.22E-03) 7.77E-01(3.78E-02) 7.83E-01(3.83E-03) 7.83E-01(3.94E-03) 7.84E-01(1.14E-03) 7.83E-01(2.35E-03)
DAS-CMaOP2(10) 5.98E-01(4.66E-02) 6.03E-01(4.83E-02)$‡$ 5.59E-01(2.40E-03) 5.66E-01(9.24E-05)$‡$ 5.37E-01(6.80E-04) 5.38E-01(2.09E-04)$‡$
DAS-CMaOP2(11) 9.52E-01(9.90E-02) 3.00E-01(2.98E-01)$†$ 1.09E-01(5.25E-02) 1.31E-01(2.24E-02)$†$ 5.48E-02(2.69E-02) 4.76E-02(4.84E-04)
DAS-CMaOP2(12) 7.72E-01(1.75E-01) 4.59E-02(1.22E-02)$†$ 8.71E-01(6.59E-03) 2.12E-02(6.13E-03)$†$ 8.78E-01(7.53E-03) 1.18E-02(2.60E-03)$†$
DAS-CMaOP3(1) 6.08E-01(1.96E-02) 6.46E-01(8.21E-03)$‡$ 4.49E-01(2.83E-02) 5.32E-01(3.65E-02)$‡$ 4.00E-01(1.14E-02) 5.39E-01(3.51E-02)$‡$
DAS-CMaOP3(2) 5.01E-01(1.40E-03) 4.98E-01(2.79E-03)$†$ 4.46E-01(2.57E-03) 4.24E-01(7.81E-03)$†$ 4.32E-01(1.23E-03) 4.21E-01(3.75E-03)$†$
DAS-CMaOP3(3) 6.26E-01(5.72E-03) 6.56E-01(9.98E-03)$‡$ 5.52E-01(5.39E-03) 5.73E-01(3.23E-02)$‡$ 5.26E-01(4.73E-03) 6.26E-01(1.87E-02)$‡$
DAS-CMaOP3(4) 4.59E-01(2.31E-03) 4.67E-01(3.64E-03)$‡$ 3.77E-01(7.59E-03) 3.87E-01(3.61E-02)$‡$ 3.65E-01(5.18E-03) 3.92E-01(1.68E-03)$‡$
DAS-CMaOP3(5) 5.94E-01(2.11E-02) 6.40E-01(1.22E-02)$‡$ 4.28E-01(3.13E-02) 5.45E-01(2.09E-02)$‡$ 3.76E-01(2.10E-02) 5.14E-01(2.97E-02)$‡$
DAS-CMaOP3(6) 4.93E-01(1.59E-03) 4.98E-01(2.14E-03)$‡$ 4.35E-01(1.37E-03) 4.34E-01(3.24E-03) 4.20E-01(1.19E-03) 4.18E-01(5.70E-03)
DAS-CMaOP3(7) 6.55E-01(8.81E-03) 6.10E-01(3.73E-02)$†$ 4.30E-01(7.21E-02) 4.71E-01(9.43E-02) 4.13E-01(1.05E-01) 3.88E-01(4.90E-02)
DAS-CMaOP3(8) 1.88E-01(5.10E-02) 2.09E-01(2.89E-03)$‡$ 8.02E-02(5.76E-02) 1.57E-01(3.54E-02)$†$ 4.58E-01(7.15E-03) 1.30E-01(1.79E-03)$†$
DAS-CMaOP3(9) 5.94E-01(1.69E-02) 6.26E-01(1.69E-02)$‡$ 4.07E-01(2.63E-02) 5.13E-01(4.50E-02)$‡$ 4.55E-01(6.59E-03) 4.95E-01(4.33E-02)$‡$
DAS-CMaOP3(10) 4.88E-01(1.94E-03) 5.00E-01(8.22E-04)$‡$ 4.26E-01(3.48E-03) 4.33E-01(5.10E-03)$‡$ 4.11E-01(1.48E-03) 4.19E-01(3.03E-03)$‡$
DAS-CMaOP3(11) 1.35E-01(3.90E-02) 3.32E-02(3.25E-04)$†$ 5.19E-01(1.05E-01) 3.61E-02(2.63E-07)$†$ 5.31E-01(2.05E-02) 4.75E-02(4.35E-05)$†$
DAS-CMaOP3(12) 4.74E-01(6.69E-03) 3.19E-02(3.67E-04)$†$ 4.53E-01(4.77E-03) 2.93E-02(1.02E-04)$†$ 4.57E-01(7.30E-03) 2.16E-02(1.03E-04)$†$
DAS-CMaOP4(1) 7.44E-01(2.53E-02) 7.54E-01(2.53E-02)$‡$ 7.61E-01(6.33E-03) 8.06E-01(5.32E-03)$‡$ 6.92E-01(1.47E-02) 8.06E-01(3.08E-03)$‡$
DAS-CMaOP4(2) 5.02E-01(8.74E-04) 5.12E-01(6.31E-04)$‡$ 4.80E-01(4.21E-03) 5.16E-01(3.65E-04)$‡$ 4.46E-01(3.66E-03) 5.16E-01(2.29E-04)$‡$
DAS-CMaOP4(3) 8.56E-01(2.36E-03) 8.60E-01(2.45E-03)$‡$ 9.10E-01(4.02E-03) 9.20E-01(1.79E-02)$‡$ 9.04E-01(5.81E-03) 9.43E-01(1.88E-02)$‡$
DAS-CMaOP4(4) 4.27E-01(1.58E-02) 4.42E-01(1.68E-02)$‡$ 4.09E-01(4.39E-03) 4.32E-01(8.14E-03)$‡$ 3.93E-01(7.52E-03) 4.27E-01(2.51E-03)$‡$
DAS-CMaOP4(5) 7.04E-01(2.54E-02) 7.19E-01(3.10E-02)$‡$ 7.45E-01(1.01E-02) 7.85E-01(9.51E-03)$‡$ 6.88E-01(1.43E-02) 7.85E-01(7.23E-03)$‡$
DAS-CMaOP4(6) 5.02E-01(5.71E-04) 5.12E-01(6.58E-04)$‡$ 4.80E-01(3.88E-03) 5.16E-01(3.30E-04)$‡$ 4.48E-01(3.96E-03) 5.16E-01(1.89E-04)$‡$
DAS-CMaOP4(7) 8.48E-01(2.70E-03) 8.46E-01(3.70E-03) 9.07E-01(3.90E-03) 8.39E-01(1.46E-02)$†$ 9.05E-01(6.69E-03) 8.86E-01(1.27E-02)$†$
DAS-CMaOP4(8) 4.18E-01(1.09E-02) 3.81E-01(4.22E-02)$†$ 3.89E-01(6.98E-03) 4.19E-01(1.26E-02)$‡$ 3.78E-01(3.42E-02) 4.13E-01(4.30E-03)$‡$
DAS-CMaOP4(9) 6.61E-01(2.57E-02) 6.38E-01(2.31E-02)$†$ 7.13E-01(1.27E-02) 7.43E-01(3.24E-02)$‡$ 6.74E-01(1.46E-02) 7.65E-01(1.01E-02)$‡$
DAS-CMaOP4(10) 5.02E-01(6.06E-04) 5.12E-01(5.06E-04)$‡$ 4.80E-01(5.00E-03) 5.16E-01(2.96E-04)$‡$ 4.47E-01(3.78E-03) 5.15E-01(2.54E-04)$‡$
DAS-CMaOP4(11) 6.96E-01(4.19E-02) 6.51E-01(1.58E-01) 2.72E-01(2.04E-02) 1.08E-01(9.64E-02)$†$ 1.57E-01(1.62E-01) 4.76E-02(4.17E-08)$†$
DAS-CMaOP4(12) 2.35E-01(6.89E-02) 2.80E-02(2.85E-03)$†$ 4.49E-01(1.59E-01) 2.44E-02(1.52E-02)$†$ 5.59E-01(1.84E-02) 1.44E-02(2.31E-05)$†$
DAS-CMaOP5(1) 7.13E-01(1.25E-02) 7.26E-01(2.33E-02)$‡$ 6.92E-01(1.20E-02) 7.67E-01(2.45E-03)$‡$ 6.07E-01(1.32E-02) 7.69E-01(3.30E-03)$‡$
DAS-CMaOP5(2) 4.89E-01(6.98E-04) 5.07E-01(8.76E-04)$‡$ 4.71E-01(4.42E-03) 5.14E-01(7.73E-04)$‡$ 4.40E-01(6.09E-03) 5.15E-01(2.47E-04)$‡$
DAS-CMaOP5(3) 8.18E-01(2.61E-03) 8.34E-01(2.24E-03)$‡$ 8.55E-01(4.33E-03) 8.97E-01(6.53E-04)$‡$ 8.50E-01(2.85E-03) 9.19E-01(7.77E-04)$‡$
DAS-CMaOP5(4) 4.26E-01(8.15E-03) 4.37E-01(1.64E-02)$‡$ 3.87E-01(6.60E-03) 4.36E-01(1.92E-03)$‡$ 3.53E-01(8.73E-03) 4.24E-01(3.13E-03)$‡$
DAS-CMaOP5(5) 6.85E-01(2.83E-02) 7.08E-01(3.09E-02)$‡$ 6.78E-01(1.01E-02) 7.56E-01(5.13E-03)$‡$ 6.08E-01(1.30E-02) 7.55E-01(4.13E-03)$‡$
DAS-CMaOP5(6) 4.90E-01(1.39E-03) 5.06E-01(1.24E-03)$‡$ 4.68E-01(6.31E-03) 5.14E-01(6.47E-04)$‡$ 4.40E-01(3.46E-03) 5.15E-01(2.48E-04)$‡$
DAS-CMaOP5(7) 8.12E-01(2.65E-03) 8.23E-01(2.54E-03)$‡$ 8.51E-01(4.07E-03) 8.86E-01(7.82E-03)$‡$ 8.47E-01(4.67E-03) 9.09E-01(7.52E-03)$‡$
DAS-CMaOP5(8) 4.12E-01(1.57E-02) 4.23E-01(1.61E-02)$‡$ 3.78E-01(5.85E-03) 4.32E-01(1.50E-03)$‡$ 3.46E-01(6.90E-03) 4.15E-01(2.82E-03)$‡$
DAS-CMaOP5(9) 6.60E-01(2.90E-02) 6.61E-01(2.90E-02) 6.55E-01(8.74E-03) 7.45E-01(4.31E-03)$‡$ 6.03E-01(1.53E-02) 7.38E-01(3.68E-03)$‡$
DAS-CMaOP5(10) 4.89E-01(1.05E-03) 5.05E-01(1.25E-03)$‡$ 4.67E-01(4.25E-03) 5.14E-01(7.68E-04)$‡$ 4.41E-01(4.98E-03) 5.14E-01(2.61E-04)$‡$
DAS-CMaOP5(11) 6.68E-01(1.27E-02) 7.75E-01(7.53E-02)$‡$ 2.71E-01(1.25E-02) 1.36E-01(8.65E-02)$†$ 1.28E-01(7.67E-03) 4.20E-02(2.68E-04)$†$
DAS-CMaOP5(12) 2.16E-01(1.91E-02) 3.56E-02(1.21E-03)$†$ 8.38E-02(9.43E-05) 4.34E-02(2.69E-02)$†$ 4.65E-01(1.76E-02) 2.61E-02(3.47E-03)$†$
DAS-CMaOP6(1) 6.44E-01(2.04E-02) 6.02E-01(3.03E-02)$†$ 7.18E-01(1.40E-02) 7.72E-01(1.28E-02)$‡$ 6.61E-01(1.22E-02) 7.77E-01(7.65E-03)$‡$
DAS-CMaOP6(2) 4.38E-01(9.40E-04) 4.42E-01(3.84E-03)$‡$ 4.77E-01(7.49E-03) 5.16E-01(2.22E-04)$‡$ 4.53E-01(5.43E-03) 5.16E-01(1.18E-04)$‡$
DAS-CMaOP6(3) 7.19E-01(5.30E-03) 6.96E-01(5.48E-03)$†$ 8.56E-01(1.44E-02) 9.02E-01(5.59E-03)$‡$ 8.47E-01(9.69E-03) 9.23E-01(5.63E-03)$‡$
DAS-CMaOP6(4) 3.76E-01(1.54E-02) 3.68E-01(2.05E-02) 4.05E-01(8.41E-03) 4.40E-01(4.17E-03)$‡$ 3.75E-01(5.99E-03) 4.29E-01(1.74E-03)$‡$
DAS-CMaOP6(5) 6.05E-01(1.94E-02) 5.73E-01(8.74E-03)$†$ 6.96E-01(1.53E-02) 7.65E-01(1.04E-02)$‡$ 6.50E-01(1.50E-02) 7.66E-01(7.38E-03)$‡$
DAS-CMaOP6(6) 4.39E-01(5.77E-04) 4.41E-01(4.44E-03)$‡$ 4.71E-01(4.43E-03) 5.16E-01(2.53E-04)$‡$ 4.45E-01(6.02E-03) 5.16E-01(1.84E-04)$‡$
DAS-CMaOP6(7) 7.24E-01(4.87E-03) 6.90E-01(4.63E-03)$†$ 8.52E-01(1.01E-02) 8.93E-01(1.17E-02)$‡$ 8.39E-01(1.21E-02) 9.04E-01(1.47E-02)$‡$
DAS-CMaOP6(8) 3.60E-01(1.38E-02) 3.49E-01(1.37E-02)$†$ 3.92E-01(8.15E-03) 4.32E-01(6.08E-03)$‡$ 3.70E-01(8.37E-03) 4.20E-01(2.22E-03)$‡$
DAS-CMaOP6(9) 5.71E-01(7.75E-03) 5.66E-01(5.09E-03)$†$ 6.79E-01(1.33E-02) 7.47E-01(9.44E-03)$‡$ 6.46E-01(1.57E-02) 7.48E-01(8.49E-03)$‡$
DAS-CMaOP6(10) 4.38E-01(5.96E-04) 4.41E-01(3.63E-03)$‡$ 4.73E-01(9.24E-03) 5.16E-01(2.31E-04)$‡$ 4.46E-01(6.64E-03) 5.16E-01(1.38E-04)$‡$
DAS-CMaOP6(11) 4.96E-01(7.13E-02) 2.53E-01(1.94E-01)$†$ 2.88E-01(4.10E-03) 5.59E-02(2.48E-02)$†$ 1.37E-01(9.77E-03) 4.38E-02(1.02E-03)$†$
DAS-CMaOP6(12) 3.87E-01(1.37E-02) 3.62E-02(1.59E-02)$†$ 8.39E-02(1.48E-04) 2.05E-02(1.65E-03)$†$ 5.39E-01(1.38E-02) 1.43E-02(4.88E-04)$†$
m $=$ 5m $=$ 8m $=$ 10
Test ProblemC-MOEA/DDC-NSGA-IIIC-MOEA/DDC-NSGA-IIIC-MOEA/DDC-NSGA-III
DAS-CMaOP1(1) 6.62E-01(3.34E-02) 6.71E-01(3.23E-02) 7.95E-01(2.54E-02) 7.63E-01(7.21E-02)$†$ 7.70E-01(4.77E-02) 8.27E-01(9.18E-03)$‡$
DAS-CMaOP1(2) 5.15E-01(7.02E-02) 5.81E-01(1.29E-02)$‡$ 4.78E-01(6.98E-02) 5.27E-01(2.19E-02)$‡$ 4.94E-01(7.05E-03) 5.00E-01(1.11E-02)$‡$
DAS-CMaOP1(3) 7.55E-01(2.56E-02) 7.67E-01(5.78E-02) 8.47E-01(1.24E-02) 8.45E-01(2.54E-02) 8.48E-01(1.72E-02) 8.63E-01(7.47E-03)$‡$
DAS-CMaOP1(4) 2.48E-01(1.35E-02) 4.16E-01(4.22E-02)$‡$ 2.83E-01(2.81E-02) 4.60E-01(1.95E-02)$‡$ 2.95E-01(2.19E-02) 4.85E-01(4.27E-03)$‡$
DAS-CMaOP1(5) 6.32E-01(3.20E-02) 6.04E-01(3.65E-02)$†$ 7.68E-01(3.94E-02) 7.09E-01(1.10E-01)$†$ 7.03E-01(6.09E-02) 8.22E-01(1.19E-02)$‡$
DAS-CMaOP1(6) 4.79E-01(5.89E-02) 5.74E-01(1.55E-02)$‡$ 3.39E-01(4.90E-02) 5.29E-01(2.15E-02)$‡$ 4.22E-01(5.90E-02) 4.95E-01(1.43E-02)$‡$
DAS-CMaOP1(7) 7.60E-01(2.18E-02) 7.53E-01(6.32E-02) 8.44E-01(2.29E-02) 8.38E-01(1.14E-02) 6.91E-01(7.33E-02) 8.65E-01(8.12E-03)$‡$
DAS-CMaOP1(8) 2.50E-01(1.52E-02) 3.36E-01(3.80E-02)$‡$ 2.63E-01(1.31E-02) 4.32E-01(6.31E-03)$‡$ 2.54E-01(1.39E-02) 3.97E-01(1.88E-03)$‡$
DAS-CMaOP1(9) 5.88E-01(3.49E-02) 5.41E-01(5.18E-02)$†$ 7.25E-01(7.12E-02) 6.59E-01(1.49E-01) 5.89E-01(8.45E-02) 8.00E-01(8.06E-02)$‡$
DAS-CMaOP1(10) 4.70E-01(5.73E-02) 5.68E-01(2.23E-02)$‡$ 3.52E-01(5.77E-02) 4.99E-01(2.24E-02)$‡$ 4.19E-01(4.25E-02) 4.89E-01(6.33E-03)$‡$
DAS-CMaOP1(11) 1.72E-01(1.81E-03) 1.84E-01(1.07E-03)$‡$ 1.06E-01(3.01E-03) 1.22E-01(1.18E-03)$‡$ 9.27E-02(1.19E-03) 9.10E-02(7.12E-04)$†$
DAS-CMaOP1(12) 2.41E-01(1.56E-02) 1.66E-01(5.86E-03)$†$ 2.48E-01(1.11E-02) 1.36E-01(4.71E-06)$†$ 2.44E-01(1.32E-02) 2.00E-01(4.25E-05)$†$
DAS-CMaOP2(1) 8.09E-01(3.54E-03) 8.10E-01(2.93E-03) 8.06E-01(5.53E-03) 8.09E-01(1.12E-03)$‡$ 8.07E-01(1.04E-03) 8.07E-01(1.28E-03)
DAS-CMaOP2(2) 6.04E-01(4.25E-02) 6.20E-01(2.64E-02)$‡$ 5.61E-01(1.67E-03) 5.66E-01(6.64E-05)$‡$ 5.37E-01(7.37E-04) 5.38E-01(6.51E-05)$‡$
DAS-CMaOP2(3) 9.61E-01(5.22E-02) 9.83E-01(3.93E-02)$‡$ 9.68E-01(3.40E-03) 9.97E-01(8.16E-04)$‡$ 9.65E-01(3.03E-03) 9.97E-01(8.92E-04)$‡$
DAS-CMaOP2(4) 5.06E-01(1.08E-03) 5.07E-01(6.87E-04)$‡$ 4.56E-01(7.10E-05) 4.56E-01(6.66E-04) 4.33E-01(4.98E-05) 4.33E-01(4.88E-04)
DAS-CMaOP2(5) 7.99E-01(4.74E-03) 8.03E-01(1.67E-03)$‡$ 8.01E-01(2.16E-03) 8.01E-01(2.38E-03) 8.01E-01(9.29E-04) 8.01E-01(1.43E-03)
DAS-CMaOP2(6) 6.15E-01(2.60E-02) 6.09E-01(4.32E-02)$†$ 5.60E-01(1.85E-03) 5.66E-01(7.16E-04)$‡$ 5.37E-01(6.81E-04) 5.38E-01(1.22E-04)$‡$
DAS-CMaOP2(7) 9.35E-01(7.55E-02) 8.94E-01(9.02E-02) 9.69E-01(3.97E-03) 9.96E-01(1.01E-03)$‡$ 9.63E-01(3.99E-03) 9.96E-01(7.62E-04)$‡$
DAS-CMaOP2(8) 4.98E-01(1.60E-03) 4.19E-01(7.53E-02)$†$ 3.95E-01(5.72E-02) 2.95E-01(9.37E-02)$‡$ 3.39E-01(4.82E-02) 2.75E-01(8.26E-02)$†$
DAS-CMaOP2(9) 7.82E-01(7.22E-03) 7.77E-01(3.78E-02) 7.83E-01(3.83E-03) 7.83E-01(3.94E-03) 7.84E-01(1.14E-03) 7.83E-01(2.35E-03)
DAS-CMaOP2(10) 5.98E-01(4.66E-02) 6.03E-01(4.83E-02)$‡$ 5.59E-01(2.40E-03) 5.66E-01(9.24E-05)$‡$ 5.37E-01(6.80E-04) 5.38E-01(2.09E-04)$‡$
DAS-CMaOP2(11) 9.52E-01(9.90E-02) 3.00E-01(2.98E-01)$†$ 1.09E-01(5.25E-02) 1.31E-01(2.24E-02)$†$ 5.48E-02(2.69E-02) 4.76E-02(4.84E-04)
DAS-CMaOP2(12) 7.72E-01(1.75E-01) 4.59E-02(1.22E-02)$†$ 8.71E-01(6.59E-03) 2.12E-02(6.13E-03)$†$ 8.78E-01(7.53E-03) 1.18E-02(2.60E-03)$†$
DAS-CMaOP3(1) 6.08E-01(1.96E-02) 6.46E-01(8.21E-03)$‡$ 4.49E-01(2.83E-02) 5.32E-01(3.65E-02)$‡$ 4.00E-01(1.14E-02) 5.39E-01(3.51E-02)$‡$
DAS-CMaOP3(2) 5.01E-01(1.40E-03) 4.98E-01(2.79E-03)$†$ 4.46E-01(2.57E-03) 4.24E-01(7.81E-03)$†$ 4.32E-01(1.23E-03) 4.21E-01(3.75E-03)$†$
DAS-CMaOP3(3) 6.26E-01(5.72E-03) 6.56E-01(9.98E-03)$‡$ 5.52E-01(5.39E-03) 5.73E-01(3.23E-02)$‡$ 5.26E-01(4.73E-03) 6.26E-01(1.87E-02)$‡$
DAS-CMaOP3(4) 4.59E-01(2.31E-03) 4.67E-01(3.64E-03)$‡$ 3.77E-01(7.59E-03) 3.87E-01(3.61E-02)$‡$ 3.65E-01(5.18E-03) 3.92E-01(1.68E-03)$‡$
DAS-CMaOP3(5) 5.94E-01(2.11E-02) 6.40E-01(1.22E-02)$‡$ 4.28E-01(3.13E-02) 5.45E-01(2.09E-02)$‡$ 3.76E-01(2.10E-02) 5.14E-01(2.97E-02)$‡$
DAS-CMaOP3(6) 4.93E-01(1.59E-03) 4.98E-01(2.14E-03)$‡$ 4.35E-01(1.37E-03) 4.34E-01(3.24E-03) 4.20E-01(1.19E-03) 4.18E-01(5.70E-03)
DAS-CMaOP3(7) 6.55E-01(8.81E-03) 6.10E-01(3.73E-02)$†$ 4.30E-01(7.21E-02) 4.71E-01(9.43E-02) 4.13E-01(1.05E-01) 3.88E-01(4.90E-02)
DAS-CMaOP3(8) 1.88E-01(5.10E-02) 2.09E-01(2.89E-03)$‡$ 8.02E-02(5.76E-02) 1.57E-01(3.54E-02)$†$ 4.58E-01(7.15E-03) 1.30E-01(1.79E-03)$†$
DAS-CMaOP3(9) 5.94E-01(1.69E-02) 6.26E-01(1.69E-02)$‡$ 4.07E-01(2.63E-02) 5.13E-01(4.50E-02)$‡$ 4.55E-01(6.59E-03) 4.95E-01(4.33E-02)$‡$
DAS-CMaOP3(10) 4.88E-01(1.94E-03) 5.00E-01(8.22E-04)$‡$ 4.26E-01(3.48E-03) 4.33E-01(5.10E-03)$‡$ 4.11E-01(1.48E-03) 4.19E-01(3.03E-03)$‡$
DAS-CMaOP3(11) 1.35E-01(3.90E-02) 3.32E-02(3.25E-04)$†$ 5.19E-01(1.05E-01) 3.61E-02(2.63E-07)$†$ 5.31E-01(2.05E-02) 4.75E-02(4.35E-05)$†$
DAS-CMaOP3(12) 4.74E-01(6.69E-03) 3.19E-02(3.67E-04)$†$ 4.53E-01(4.77E-03) 2.93E-02(1.02E-04)$†$ 4.57E-01(7.30E-03) 2.16E-02(1.03E-04)$†$
DAS-CMaOP4(1) 7.44E-01(2.53E-02) 7.54E-01(2.53E-02)$‡$ 7.61E-01(6.33E-03) 8.06E-01(5.32E-03)$‡$ 6.92E-01(1.47E-02) 8.06E-01(3.08E-03)$‡$
DAS-CMaOP4(2) 5.02E-01(8.74E-04) 5.12E-01(6.31E-04)$‡$ 4.80E-01(4.21E-03) 5.16E-01(3.65E-04)$‡$ 4.46E-01(3.66E-03) 5.16E-01(2.29E-04)$‡$
DAS-CMaOP4(3) 8.56E-01(2.36E-03) 8.60E-01(2.45E-03)$‡$ 9.10E-01(4.02E-03) 9.20E-01(1.79E-02)$‡$ 9.04E-01(5.81E-03) 9.43E-01(1.88E-02)$‡$
DAS-CMaOP4(4) 4.27E-01(1.58E-02) 4.42E-01(1.68E-02)$‡$ 4.09E-01(4.39E-03) 4.32E-01(8.14E-03)$‡$ 3.93E-01(7.52E-03) 4.27E-01(2.51E-03)$‡$
DAS-CMaOP4(5) 7.04E-01(2.54E-02) 7.19E-01(3.10E-02)$‡$ 7.45E-01(1.01E-02) 7.85E-01(9.51E-03)$‡$ 6.88E-01(1.43E-02) 7.85E-01(7.23E-03)$‡$
DAS-CMaOP4(6) 5.02E-01(5.71E-04) 5.12E-01(6.58E-04)$‡$ 4.80E-01(3.88E-03) 5.16E-01(3.30E-04)$‡$ 4.48E-01(3.96E-03) 5.16E-01(1.89E-04)$‡$
DAS-CMaOP4(7) 8.48E-01(2.70E-03) 8.46E-01(3.70E-03) 9.07E-01(3.90E-03) 8.39E-01(1.46E-02)$†$ 9.05E-01(6.69E-03) 8.86E-01(1.27E-02)$†$
DAS-CMaOP4(8) 4.18E-01(1.09E-02) 3.81E-01(4.22E-02)$†$ 3.89E-01(6.98E-03) 4.19E-01(1.26E-02)$‡$ 3.78E-01(3.42E-02) 4.13E-01(4.30E-03)$‡$
DAS-CMaOP4(9) 6.61E-01(2.57E-02) 6.38E-01(2.31E-02)$†$ 7.13E-01(1.27E-02) 7.43E-01(3.24E-02)$‡$ 6.74E-01(1.46E-02) 7.65E-01(1.01E-02)$‡$
DAS-CMaOP4(10) 5.02E-01(6.06E-04) 5.12E-01(5.06E-04)$‡$ 4.80E-01(5.00E-03) 5.16E-01(2.96E-04)$‡$ 4.47E-01(3.78E-03) 5.15E-01(2.54E-04)$‡$
DAS-CMaOP4(11) 6.96E-01(4.19E-02) 6.51E-01(1.58E-01) 2.72E-01(2.04E-02) 1.08E-01(9.64E-02)$†$ 1.57E-01(1.62E-01) 4.76E-02(4.17E-08)$†$
DAS-CMaOP4(12) 2.35E-01(6.89E-02) 2.80E-02(2.85E-03)$†$ 4.49E-01(1.59E-01) 2.44E-02(1.52E-02)$†$ 5.59E-01(1.84E-02) 1.44E-02(2.31E-05)$†$
DAS-CMaOP5(1) 7.13E-01(1.25E-02) 7.26E-01(2.33E-02)$‡$ 6.92E-01(1.20E-02) 7.67E-01(2.45E-03)$‡$ 6.07E-01(1.32E-02) 7.69E-01(3.30E-03)$‡$
DAS-CMaOP5(2) 4.89E-01(6.98E-04) 5.07E-01(8.76E-04)$‡$ 4.71E-01(4.42E-03) 5.14E-01(7.73E-04)$‡$ 4.40E-01(6.09E-03) 5.15E-01(2.47E-04)$‡$
DAS-CMaOP5(3) 8.18E-01(2.61E-03) 8.34E-01(2.24E-03)$‡$ 8.55E-01(4.33E-03) 8.97E-01(6.53E-04)$‡$ 8.50E-01(2.85E-03) 9.19E-01(7.77E-04)$‡$
DAS-CMaOP5(4) 4.26E-01(8.15E-03) 4.37E-01(1.64E-02)$‡$ 3.87E-01(6.60E-03) 4.36E-01(1.92E-03)$‡$ 3.53E-01(8.73E-03) 4.24E-01(3.13E-03)$‡$
DAS-CMaOP5(5) 6.85E-01(2.83E-02) 7.08E-01(3.09E-02)$‡$ 6.78E-01(1.01E-02) 7.56E-01(5.13E-03)$‡$ 6.08E-01(1.30E-02) 7.55E-01(4.13E-03)$‡$
DAS-CMaOP5(6) 4.90E-01(1.39E-03) 5.06E-01(1.24E-03)$‡$ 4.68E-01(6.31E-03) 5.14E-01(6.47E-04)$‡$ 4.40E-01(3.46E-03) 5.15E-01(2.48E-04)$‡$
DAS-CMaOP5(7) 8.12E-01(2.65E-03) 8.23E-01(2.54E-03)$‡$ 8.51E-01(4.07E-03) 8.86E-01(7.82E-03)$‡$ 8.47E-01(4.67E-03) 9.09E-01(7.52E-03)$‡$
DAS-CMaOP5(8) 4.12E-01(1.57E-02) 4.23E-01(1.61E-02)$‡$ 3.78E-01(5.85E-03) 4.32E-01(1.50E-03)$‡$ 3.46E-01(6.90E-03) 4.15E-01(2.82E-03)$‡$
DAS-CMaOP5(9) 6.60E-01(2.90E-02) 6.61E-01(2.90E-02) 6.55E-01(8.74E-03) 7.45E-01(4.31E-03)$‡$ 6.03E-01(1.53E-02) 7.38E-01(3.68E-03)$‡$
DAS-CMaOP5(10) 4.89E-01(1.05E-03) 5.05E-01(1.25E-03)$‡$ 4.67E-01(4.25E-03) 5.14E-01(7.68E-04)$‡$ 4.41E-01(4.98E-03) 5.14E-01(2.61E-04)$‡$
DAS-CMaOP5(11) 6.68E-01(1.27E-02) 7.75E-01(7.53E-02)$‡$ 2.71E-01(1.25E-02) 1.36E-01(8.65E-02)$†$ 1.28E-01(7.67E-03) 4.20E-02(2.68E-04)$†$
DAS-CMaOP5(12) 2.16E-01(1.91E-02) 3.56E-02(1.21E-03)$†$ 8.38E-02(9.43E-05) 4.34E-02(2.69E-02)$†$ 4.65E-01(1.76E-02) 2.61E-02(3.47E-03)$†$
DAS-CMaOP6(1) 6.44E-01(2.04E-02) 6.02E-01(3.03E-02)$†$ 7.18E-01(1.40E-02) 7.72E-01(1.28E-02)$‡$ 6.61E-01(1.22E-02) 7.77E-01(7.65E-03)$‡$
DAS-CMaOP6(2) 4.38E-01(9.40E-04) 4.42E-01(3.84E-03)$‡$ 4.77E-01(7.49E-03) 5.16E-01(2.22E-04)$‡$ 4.53E-01(5.43E-03) 5.16E-01(1.18E-04)$‡$
DAS-CMaOP6(3) 7.19E-01(5.30E-03) 6.96E-01(5.48E-03)$†$ 8.56E-01(1.44E-02) 9.02E-01(5.59E-03)$‡$ 8.47E-01(9.69E-03) 9.23E-01(5.63E-03)$‡$
DAS-CMaOP6(4) 3.76E-01(1.54E-02) 3.68E-01(2.05E-02) 4.05E-01(8.41E-03) 4.40E-01(4.17E-03)$‡$ 3.75E-01(5.99E-03) 4.29E-01(1.74E-03)$‡$
DAS-CMaOP6(5) 6.05E-01(1.94E-02) 5.73E-01(8.74E-03)$†$ 6.96E-01(1.53E-02) 7.65E-01(1.04E-02)$‡$ 6.50E-01(1.50E-02) 7.66E-01(7.38E-03)$‡$
DAS-CMaOP6(6) 4.39E-01(5.77E-04) 4.41E-01(4.44E-03)$‡$ 4.71E-01(4.43E-03) 5.16E-01(2.53E-04)$‡$ 4.45E-01(6.02E-03) 5.16E-01(1.84E-04)$‡$
DAS-CMaOP6(7) 7.24E-01(4.87E-03) 6.90E-01(4.63E-03)$†$ 8.52E-01(1.01E-02) 8.93E-01(1.17E-02)$‡$ 8.39E-01(1.21E-02) 9.04E-01(1.47E-02)$‡$
DAS-CMaOP6(8) 3.60E-01(1.38E-02) 3.49E-01(1.37E-02)$†$ 3.92E-01(8.15E-03) 4.32E-01(6.08E-03)$‡$ 3.70E-01(8.37E-03) 4.20E-01(2.22E-03)$‡$
DAS-CMaOP6(9) 5.71E-01(7.75E-03) 5.66E-01(5.09E-03)$†$ 6.79E-01(1.33E-02) 7.47E-01(9.44E-03)$‡$ 6.46E-01(1.57E-02) 7.48E-01(8.49E-03)$‡$
DAS-CMaOP6(10) 4.38E-01(5.96E-04) 4.41E-01(3.63E-03)$‡$ 4.73E-01(9.24E-03) 5.16E-01(2.31E-04)$‡$ 4.46E-01(6.64E-03) 5.16E-01(1.38E-04)$‡$
DAS-CMaOP6(11) 4.96E-01(7.13E-02) 2.53E-01(1.94E-01)$†$ 2.88E-01(4.10E-03) 5.59E-02(2.48E-02)$†$ 1.37E-01(9.77E-03) 4.38E-02(1.02E-03)$†$
DAS-CMaOP6(12) 3.87E-01(1.37E-02) 3.62E-02(1.59E-02)$†$ 8.39E-02(1.48E-04) 2.05E-02(1.65E-03)$†$ 5.39E-01(1.38E-02) 1.43E-02(4.88E-04)$†$

For 5-objective DAS-CMaOP2 with each difficulty level less than 0.5, C-NSGA-III is better or significantly better than C-MOEA/DD. For 5-objective DAS-CMaOP2 with the remainder of the difficulty triplets, C-MOEA/DD is better or significantly better than C-NSGA-III except for the cases with difficulty triplets $(0.5,0.0,0.0)$ and $(0.0,0.75,0.0)$. For 8-objective DAS-CMaOP2, C-NSGA-III is significantly better than C-MOEA/DD in most cases. For 10-objective DAS-CMaOP2 with feasibility-hardness, C-NSGA-III is significantly better than C-MOEA/DD. For 5-objective DAS-CMaOP3, C-NSGA-III is significantly better than C-MOEA/DD in most cases. For 8- and 10-objective DAS-CMaOP3 with diversity-hardness, C-NSGA-III is also significantly better than C-MOEA/DD.

For 5-, 8-, and 10-objective DAS-CMaOP4 and DAS-CMaOP5, C-NSGA-III is significantly better than C-MOEA/DD in most cases in $HV$ values, and C-MOEA/DD is significantly better than C-NSGA-III in the case with the difficulty triplet $(0.75,0.75,0.75)$. For 5-objective DAS-CMaOP6 with feasibility-hardness, C-NSGA-III is significantly better than C-MOEA/DD, and for 5-objective DAS-CMaOP6 with the remaining difficulty triplets, C-MOEA/DD is better or significantly better than C-NSGA-III. For 8- and 10-objective DAS-CMaOP6 with difficulty triplets $(0.0,0.0,0.75)$ and $(0.75,0.75,0.75)$, C-MOEA/DD is significantly better than C-NSGA-III. For 8- and 10-objective DAS-CMaOP6 with the remaining difficulty triplets, C-NSGA-III is significantly better than C-MOEA/DD.

Table 7 presents the statistics on $HV$ values for C-MOEA/DD and C-NSGA-III on 5-, 8-, and 10-objective DAS-CMaOP6-9 with the first twelve difficulty triplets in Table 3. We can see that for 5-, 8-, and 10-objective DAS-CMaOP7 with feasibility-hardness, C-NSGA-III is significantly better than C-MOEA/DD. For 5-objective DAS-CMaOP7 with simultaneous diversity-, feasibility-, and convergence-hardness—that is, the difficulty triplets are (0.25, 0.25, 0.25), (0.5, 0.5, 0.5), (0.75, 0.75, 0.75)—C-MOEA/DD is significantly better than C-NSGA-III. For 8- and 10-objective DAS-CMaOP7 with difficulty triplets $(0.0,0.0,0.75)$ and $(0.75,0.75,0.75)$, C-MOEA/DD is significantly better than C-NSGA-III.

Table 7:
Means and standard deviations of $HV$ values obtained by C-MOEA/DD and C-NSGA-III on DAS-CMaOP7-9 with 5, 8, and 10 objectives. Wilcoxon's rank sum test at 0.05 significance level is performed between C-MOEA/DD and C-NSGA-III. $†$ and $‡$ denote that the performance of C-NSGA-III is significantly worse than or better than that of C-MOEA/DD, respectively. DAS-CMaOP7(i) means that DAS-CMaOP7 has the $i$-th difficulty triplet in Table 3.
m $=$ 5m $=$ 8m $=$ 10
Test ProblemC-MOEA/DDC-NSGA-IIIC-MOEA/DDC-NSGA-IIIC-MOEA/DDC-NSGA-III
DAS-CMaOP7(1) 7.31E-01(2.43E-02) 7.40E-01(3.47E-02) 7.38E-01(1.71E-02) 7.70E-01(7.75E-02)$‡$ 7.13E-01(2.50E-02) 7.14E-01(1.27E-01)
DAS-CMaOP7(2) 5.01E-01(5.83E-04) 5.11E-01(4.97E-04)$‡$ 4.76E-01(3.43E-03) 5.14E-01(2.83E-04)$‡$ 4.48E-01(6.58E-03) 5.14E-01(3.42E-04)$‡$
DAS-CMaOP7(3) 8.66E-01(1.69E-03) 8.76E-01(1.38E-03)$‡$ 9.07E-01(5.59E-03) 9.39E-01(7.74E-03)$‡$ 9.06E-01(8.64E-03) 9.66E-01(8.03E-03)$‡$
DAS-CMaOP7(4) 4.22E-01(1.59E-02) 4.18E-01(1.86E-02) 3.97E-01(1.05E-02) 3.99E-01(4.88E-02)$‡$ 3.76E-01(1.01E-02) 3.88E-01(3.92E-02)$‡$
DAS-CMaOP7(5) 7.09E-01(3.11E-02) 6.82E-01(3.46E-02)$†$ 7.17E-01(1.40E-02) 6.56E-01(1.54E-01) 6.92E-01(1.89E-02) 5.70E-01(1.74E-01)
DAS-CMaOP7(6) 5.01E-01(4.07E-04) 5.11E-01(4.26E-04)$‡$ 4.74E-01(4.96E-03) 5.14E-01(3.85E-04)$‡$ 4.45E-01(6.42E-03) 5.14E-01(2.27E-04)$‡$
DAS-CMaOP7(7) 8.61E-01(1.80E-03) 8.70E-01(1.68E-03)$‡$ 9.00E-01(7.98E-03) 9.24E-01(8.93E-03)$‡$ 9.00E-01(9.04E-03) 9.47E-01(1.22E-02)$‡$
DAS-CMaOP7(8) 4.13E-01(1.69E-02) 3.27E-01(6.29E-02)$†$ 3.80E-01(1.13E-02) 2.62E-01(7.02E-02)$†$ 3.60E-01(9.87E-03) 3.17E-01(8.34E-02)
DAS-CMaOP7(9) 6.71E-01(2.74E-02) 5.56E-01(1.19E-01)$†$ 6.72E-01(1.77E-02) 3.92E-01(1.62E-01)$†$ 6.52E-01(1.97E-02) 3.75E-01(1.22E-01)$†$
DAS-CMaOP7(10) 5.01E-01(5.83E-04) 5.11E-01(3.90E-04)$‡$ 4.76E-01(5.88E-03) 5.14E-01(4.22E-04)$‡$ 4.44E-01(6.12E-03) 5.14E-01(2.64E-04)$‡$
DAS-CMaOP7(11) 6.24E-01(3.45E-02) 5.72E-01(6.82E-02)$†$ 2.76E-01(2.17E-02) 3.61E-02(6.84E-11)$†$ 1.26E-01(6.84E-03) 4.76E-02(1.72E-07)$†$
DAS-CMaOP7(12) 1.88E-01(2.93E-02) 2.19E-02(3.33E-05)$†$ 1.68E-01(1.73E-01) 2.10E-02(5.63E-05)$†$ 5.40E-01(1.78E-02) 1.44E-02(2.30E-05)$†$
DAS-CMaOP8(1) 6.74E-01(1.79E-02) 6.62E-01(2.87E-02) 7.28E-01(6.03E-03) 7.37E-01(4.23E-03)$‡$ 7.64E-01(4.02E-03) 7.77E-01(3.45E-03)$‡$
DAS-CMaOP8(2) 5.02E-01(1.90E-03) 5.13E-01(6.55E-04)$‡$ 4.89E-01(5.11E-03) 5.14E-01(7.99E-03)$‡$ 4.57E-01(4.37E-03) 5.15E-01(4.12E-03)$‡$
DAS-CMaOP8(3) 7.68E-01(1.26E-02) 7.69E-01(3.35E-03)$‡$ 8.36E-01(4.39E-03) 8.51E-01(2.02E-03)$‡$ 8.89E-01(3.96E-03) 9.06E-01(3.75E-03)$‡$
DAS-CMaOP8(4) 4.34E-01(1.14E-02) 4.30E-01(2.15E-02) 4.16E-01(5.20E-03) 4.40E-01(2.60E-03)$‡$ 4.04E-01(1.13E-02) 4.29E-01(2.16E-03)$‡$
DAS-CMaOP8(5) 6.63E-01(1.50E-02) 6.32E-01(2.88E-02)$†$ 7.07E-01(3.16E-02) 7.29E-01(6.89E-03)$‡$ 7.47E-01(1.42E-02) 7.68E-01(3.73E-03)$‡$
DAS-CMaOP8(6) 5.03E-01(5.26E-04) 5.13E-01(7.76E-04)$‡$ 4.93E-01(2.89E-03) 5.08E-01(8.78E-03)$‡$ 4.66E-01(7.78E-03) 5.10E-01(5.31E-03)$‡$
DAS-CMaOP8(7) 7.70E-01(1.24E-02) 7.58E-01(6.65E-03)$†$ 8.41E-01(1.18E-02) 7.61E-01(1.88E-02)$†$ 8.84E-01(1.24E-02) 8.22E-01(4.17E-02)$†$
DAS-CMaOP8(8) 4.20E-01(1.36E-02) 3.91E-01(2.42E-02)$†$ 4.02E-01(1.53E-02) 4.26E-01(4.82E-03)$‡$ 4.02E-01(3.44E-03) 4.19E-01(3.00E-03)$‡$
DAS-CMaOP8(9) 6.36E-01(2.51E-02) 5.84E-01(2.80E-02)$†$ 6.74E-01(5.14E-02) 7.08E-01(1.72E-02)$‡$ 7.19E-01(3.48E-02) 7.55E-01(4.33E-03)$‡$
DAS-CMaOP8(10) 5.02E-01(1.46E-03) 4.84E-01(2.18E-02)$†$ 4.79E-01(8.86E-03) 4.98E-01(6.15E-03)$‡$ 4.93E-01(2.35E-03) 5.04E-01(1.91E-03)$‡$
DAS-CMaOP8(11) 5.37E-01(3.47E-02) 3.88E-01(2.08E-01)$†$ 2.73E-01(1.26E-02) 3.61E-02(6.26E-08)$†$ 1.41E-01(3.27E-03) 5.25E-02(2.18E-02)$†$
DAS-CMaOP8(12) 2.21E-01(2.49E-02) 2.78E-02(4.23E-03)$†$ 4.83E-01(1.18E-02) 2.11E-02(2.14E-06)$†$ 5.24E-01(1.14E-02) 1.44E-02(1.12E-06)$†$
DAS-CMaOP9(1) 5.64E-01(1.16E-02) 5.80E-01(1.45E-02)$‡$ 6.39E-01(1.73E-02) 7.19E-01(2.62E-02)$‡$ 5.87E-01(2.17E-02) 7.11E-01(2.19E-02)$‡$
DAS-CMaOP9(2) 4.04E-01(1.93E-03) 4.02E-01(4.00E-03)$†$ 4.45E-01(5.55E-03) 4.99E-01(1.64E-03)$‡$ 4.34E-01(6.10E-03) 5.01E-01(1.32E-03)$‡$
DAS-CMaOP9(3) 6.36E-01(1.23E-02) 6.52E-01(1.35E-02)$†$ 8.01E-01(9.72E-03) 8.80E-01(1.75E-02)$‡$ 7.91E-01(1.59E-02) 9.08E-01(9.63E-03)$‡$
DAS-CMaOP9(4) 3.45E-01(3.42E-03) 3.45E-01(5.41E-03) 3.68E-01(7.64E-03) 4.04E-01(6.69E-03)$‡$ 3.57E-01(1.18E-02) 3.92E-01(4.99E-03)$‡$
DAS-CMaOP9(5) 5.48E-01(1.22E-02) 5.68E-01(1.00E-02)$‡$ 6.10E-01(2.65E-02) 6.89E-01(1.94E-02)$‡$ 4.27E-01(6.90E-02) 6.98E-01(3.41E-02)$‡$
DAS-CMaOP9(6) 4.06E-01(1.50E-03) 4.01E-01(2.70E-03)$†$ 4.46E-01(7.64E-03) 4.99E-01(1.67E-03)$‡$ 4.24E-01(5.74E-03) 5.02E-01(7.63E-04)$‡$
DAS-CMaOP9(7) 6.45E-01(1.45E-02) 6.35E-01(9.99E-03)$†$ 7.85E-01(1.91E-02) 8.22E-01(1.64E-02)$‡$ 7.71E-01(2.12E-02) 8.64E-01(1.42E-02)$‡$
DAS-CMaOP9(8) 3.29E-01(6.23E-03) 3.26E-01(9.66E-03) 3.40E-01(1.35E-02) 3.92E-01(1.47E-02)$‡$ 4.12E-01(2.58E-02) 3.53E-01(3.36E-02)$†$
DAS-CMaOP9(9) 5.08E-01(2.98E-02) 5.26E-01(3.26E-02) 4.87E-01(6.41E-02) 6.32E-01(5.59E-02)$‡$ 4.19E-01(3.72E-02) 6.58E-01(2.48E-02)$‡$
DAS-CMaOP9(10) 4.05E-01(1.77E-03) 4.06E-01(2.90E-03) 4.45E-01(1.21E-02) 4.99E-01(1.73E-03)$‡$ 4.25E-01(8.70E-03) 5.02E-01(1.06E-03)$‡$
DAS-CMaOP9(11) 2.74E-01(3.89E-02) 2.08E-01(1.01E-01) 2.39E-01(9.42E-03) 5.72E-02(4.19E-02)$†$ 1.19E-01(4.03E-03) 3.80E-02(1.34E-06)$†$
DAS-CMaOP9(12) 3.29E-01(1.66E-02) 2.34E-02(1.99E-03)$†$ 4.14E-01(3.55E-02) 2.10E-02(1.50E-03)$†$ 4.14E-01(2.54E-02) 1.69E-02(1.35E-03)$†$
m $=$ 5m $=$ 8m $=$ 10
Test ProblemC-MOEA/DDC-NSGA-IIIC-MOEA/DDC-NSGA-IIIC-MOEA/DDC-NSGA-III
DAS-CMaOP7(1) 7.31E-01(2.43E-02) 7.40E-01(3.47E-02) 7.38E-01(1.71E-02) 7.70E-01(7.75E-02)$‡$ 7.13E-01(2.50E-02) 7.14E-01(1.27E-01)
DAS-CMaOP7(2) 5.01E-01(5.83E-04) 5.11E-01(4.97E-04)$‡$ 4.76E-01(3.43E-03) 5.14E-01(2.83E-04)$‡$ 4.48E-01(6.58E-03) 5.14E-01(3.42E-04)$‡$
DAS-CMaOP7(3) 8.66E-01(1.69E-03) 8.76E-01(1.38E-03)$‡$ 9.07E-01(5.59E-03) 9.39E-01(7.74E-03)$‡$ 9.06E-01(8.64E-03) 9.66E-01(8.03E-03)$‡$
DAS-CMaOP7(4) 4.22E-01(1.59E-02) 4.18E-01(1.86E-02) 3.97E-01(1.05E-02) 3.99E-01(4.88E-02)$‡$ 3.76E-01(1.01E-02) 3.88E-01(3.92E-02)$‡$
DAS-CMaOP7(5) 7.09E-01(3.11E-02) 6.82E-01(3.46E-02)$†$ 7.17E-01(1.40E-02) 6.56E-01(1.54E-01) 6.92E-01(1.89E-02) 5.70E-01(1.74E-01)
DAS-CMaOP7(6) 5.01E-01(4.07E-04) 5.11E-01(4.26E-04)$‡$ 4.74E-01(4.96E-03) 5.14E-01(3.85E-04)$‡$ 4.45E-01(6.42E-03) 5.14E-01(2.27E-04)$‡$
DAS-CMaOP7(7) 8.61E-01(1.80E-03) 8.70E-01(1.68E-03)$‡$ 9.00E-01(7.98E-03) 9.24E-01(8.93E-03)$‡$ 9.00E-01(9.04E-03) 9.47E-01(1.22E-02)$‡$
DAS-CMaOP7(8) 4.13E-01(1.69E-02) 3.27E-01(6.29E-02)$†$ 3.80E-01(1.13E-02) 2.62E-01(7.02E-02)$†$ 3.60E-01(9.87E-03) 3.17E-01(8.34E-02)
DAS-CMaOP7(9) 6.71E-01(2.74E-02) 5.56E-01(1.19E-01)$†$ 6.72E-01(1.77E-02) 3.92E-01(1.62E-01)$†$ 6.52E-01(1.97E-02) 3.75E-01(1.22E-01)$†$
DAS-CMaOP7(10) 5.01E-01(5.83E-04) 5.11E-01(3.90E-04)$‡$ 4.76E-01(5.88E-03) 5.14E-01(4.22E-04)$‡$ 4.44E-01(6.12E-03) 5.14E-01(2.64E-04)$‡$
DAS-CMaOP7(11) 6.24E-01(3.45E-02) 5.72E-01(6.82E-02)$†$ 2.76E-01(2.17E-02) 3.61E-02(6.84E-11)$†$ 1.26E-01(6.84E-03) 4.76E-02(1.72E-07)$†$
DAS-CMaOP7(12) 1.88E-01(2.93E-02) 2.19E-02(3.33E-05)$†$ 1.68E-01(1.73E-01) 2.10E-02(5.63E-05)$†$ 5.40E-01(1.78E-02) 1.44E-02(2.30E-05)$†$
DAS-CMaOP8(1) 6.74E-01(1.79E-02) 6.62E-01(2.87E-02) 7.28E-01(6.03E-03) 7.37E-01(4.23E-03)$‡$ 7.64E-01(4.02E-03) 7.77E-01(3.45E-03)$‡$
DAS-CMaOP8(2) 5.02E-01(1.90E-03) 5.13E-01(6.55E-04)$‡$ 4.89E-01(5.11E-03) 5.14E-01(7.99E-03)$‡$ 4.57E-01(4.37E-03) 5.15E-01(4.12E-03)$‡$
DAS-CMaOP8(3) 7.68E-01(1.26E-02) 7.69E-01(3.35E-03)$‡$ 8.36E-01(4.39E-03) 8.51E-01(2.02E-03)$‡$ 8.89E-01(3.96E-03) 9.06E-01(3.75E-03)$‡$
DAS-CMaOP8(4) 4.34E-01(1.14E-02) 4.30E-01(2.15E-02) 4.16E-01(5.20E-03) 4.40E-01(2.60E-03)$‡$ 4.04E-01(1.13E-02) 4.29E-01(2.16E-03)$‡$
DAS-CMaOP8(5) 6.63E-01(1.50E-02) 6.32E-01(2.88E-02)$†$ 7.07E-01(3.16E-02) 7.29E-01(6.89E-03)$‡$ 7.47E-01(1.42E-02) 7.68E-01(3.73E-03)$‡$
DAS-CMaOP8(6) 5.03E-01(5.26E-04) 5.13E-01(7.76E-04)$‡$ 4.93E-01(2.89E-03) 5.08E-01(8.78E-03)$‡$ 4.66E-01(7.78E-03) 5.10E-01(5.31E-03)$‡$
DAS-CMaOP8(7) 7.70E-01(1.24E-02) 7.58E-01(6.65E-03)$†$ 8.41E-01(1.18E-02) 7.61E-01(1.88E-02)$†$ 8.84E-01(1.24E-02) 8.22E-01(4.17E-02)$†$
DAS-CMaOP8(8) 4.20E-01(1.36E-02) 3.91E-01(2.42E-02)$†$ 4.02E-01(1.53E-02) 4.26E-01(4.82E-03)$‡$ 4.02E-01(3.44E-03) 4.19E-01(3.00E-03)$‡$
DAS-CMaOP8(9) 6.36E-01(2.51E-02) 5.84E-01(2.80E-02)$†$ 6.74E-01(5.14E-02) 7.08E-01(1.72E-02)$‡$ 7.19E-01(3.48E-02) 7.55E-01(4.33E-03)$‡$
DAS-CMaOP8(10) 5.02E-01(1.46E-03) 4.84E-01(2.18E-02)$†$ 4.79E-01(8.86E-03) 4.98E-01(6.15E-03)$‡$ 4.93E-01(2.35E-03) 5.04E-01(1.91E-03)$‡$
DAS-CMaOP8(11) 5.37E-01(3.47E-02) 3.88E-01(2.08E-01)$†$ 2.73E-01(1.26E-02) 3.61E-02(6.26E-08)$†$ 1.41E-01(3.27E-03) 5.25E-02(2.18E-02)$†$
DAS-CMaOP8(12) 2.21E-01(2.49E-02) 2.78E-02(4.23E-03)$†$ 4.83E-01(1.18E-02) 2.11E-02(2.14E-06)$†$ 5.24E-01(1.14E-02) 1.44E-02(1.12E-06)$†$
DAS-CMaOP9(1) 5.64E-01(1.16E-02) 5.80E-01(1.45E-02)$‡$ 6.39E-01(1.73E-02) 7.19E-01(2.62E-02)$‡$ 5.87E-01(2.17E-02) 7.11E-01(2.19E-02)$‡$
DAS-CMaOP9(2) 4.04E-01(1.93E-03) 4.02E-01(4.00E-03)$†$ 4.45E-01(5.55E-03) 4.99E-01(1.64E-03)$‡$ 4.34E-01(6.10E-03) 5.01E-01(1.32E-03)$‡$
DAS-CMaOP9(3) 6.36E-01(1.23E-02) 6.52E-01(1.35E-02)$†$ 8.01E-01(9.72E-03) 8.80E-01(1.75E-02)$‡$ 7.91E-01(1.59E-02) 9.08E-01(9.63E-03)$‡$
DAS-CMaOP9(4) 3.45E-01(3.42E-03) 3.45E-01(5.41E-03) 3.68E-01(7.64E-03) 4.04E-01(6.69E-03)$‡$ 3.57E-01(1.18E-02) 3.92E-01(4.99E-03)$‡$
DAS-CMaOP9(5) 5.48E-01(1.22E-02) 5.68E-01(1.00E-02)$‡$ 6.10E-01(2.65E-02) 6.89E-01(1.94E-02)$‡$ 4.27E-01(6.90E-02) 6.98E-01(3.41E-02)$‡$
DAS-CMaOP9(6) 4.06E-01(1.50E-03) 4.01E-01(2.70E-03)$†$ 4.46E-01(7.64E-03) 4.99E-01(1.67E-03)$‡$ 4.24E-01(5.74E-03) 5.02E-01(7.63E-04)$‡$
DAS-CMaOP9(7) 6.45E-01(1.45E-02) 6.35E-01(9.99E-03)$†$ 7.85E-01(1.91E-02) 8.22E-01(1.64E-02)$‡$ 7.71E-01(2.12E-02) 8.64E-01(1.42E-02)$‡$
DAS-CMaOP9(8) 3.29E-01(6.23E-03) 3.26E-01(9.66E-03) 3.40E-01(1.35E-02) 3.92E-01(1.47E-02)$‡$ 4.12E-01(2.58E-02) 3.53E-01(3.36E-02)$†$
DAS-CMaOP9(9) 5.08E-01(2.98E-02) 5.26E-01(3.26E-02) 4.87E-01(6.41E-02) 6.32E-01(5.59E-02)$‡$ 4.19E-01(3.72E-02) 6.58E-01(2.48E-02)$‡$
DAS-CMaOP9(10) 4.05E-01(1.77E-03) 4.06E-01(2.90E-03) 4.45E-01(1.21E-02) 4.99E-01(1.73E-03)$‡$ 4.25E-01(8.70E-03) 5.02E-01(1.06E-03)$‡$
DAS-CMaOP9(11) 2.74E-01(3.89E-02) 2.08E-01(1.01E-01) 2.39E-01(9.42E-03) 5.72E-02(4.19E-02)$†$ 1.19E-01(4.03E-03) 3.80E-02(1.34E-06)$†$
DAS-CMaOP9(12) 3.29E-01(1.66E-02) 2.34E-02(1.99E-03)$†$ 4.14E-01(3.55E-02) 2.10E-02(1.50E-03)$†$ 4.14E-01(2.54E-02) 1.69E-02(1.35E-03)$†$

For 5-objective DAS-CMaOP8, C-MOEA/DD is significantly better than C-NSGA-III in most cases in $HV$ values. For 8- and 10-objective DAS-CMaOP8, C-NSGA-III is significantly better than C-MOEA/DD in most cases. For 8- and 10-objective DAS-CMaOP8 with difficulty triplets $(0.0,0.0,0.75)$ and $(0.75,0.75,0.75)$, C-MOEA/DD is significantly better than C-NSGA-III.

For 5-objective DAS-CMaOP9 with diversity-hardness, C-NSGA-III is significantly better than C-MOEA/DD, and for 5-objective DAS-CMaOP9 with convergence-hardness, C-MOEA/DD is better or significantly better than C-NSGA-III. For 8- and 10-objective DAS-CMaOP9, C-NSGA-III is significantly better than C-MOEA/DD in most cases. For 8- and 10-objective DAS-CMaOP9 with difficulty triplets $(0.0,0.0,0.75)$ and $(0.75,0.75,0.75)$, C-MOEA/DD is significantly better than C-NSGA-III.

Figure 8 shows the median $HV$ values of the feasible and nondominated solutions in 20 independent runs by using C-MOEA/DD and C-NSGA-III on DAS-CMaOP1-3, DAS-CMaOP5, and DAS-CMaOP8 with different difficulty triplets. We can see that according to the $HV$ measure, C-NSGA-III is more suitable for solving feasibility-hard DAS-CMaOPs, while C-MOEA/DD is more suitable for solving convergence-hard DAS-CMaOPs.

Figure 8:

Parallel coordinate plots of the feasible and nondominated solutions with the median $HV$ values in 20 independent runs using C-MOEA/DD and C-NSGA-III on DAS-CMaOP1-3, DAS-CMaOP5 and DAS-CMaOP8 with different difficulty triplets. (a)–(f) show the populations achieved by C-MOEA/DD and C-NSGA-III on DAS-CMaOPs with feasibility-hardness. (g)–(l) show the populations achieved by C-MOEA/DD and C-NSGA-III on DAS-CMOPs with convergence-hardness. It is worth noting that no feasible solutions have been found in (c) and (l).

Figure 8:

Parallel coordinate plots of the feasible and nondominated solutions with the median $HV$ values in 20 independent runs using C-MOEA/DD and C-NSGA-III on DAS-CMaOP1-3, DAS-CMaOP5 and DAS-CMaOP8 with different difficulty triplets. (a)–(f) show the populations achieved by C-MOEA/DD and C-NSGA-III on DAS-CMaOPs with feasibility-hardness. (g)–(l) show the populations achieved by C-MOEA/DD and C-NSGA-III on DAS-CMOPs with convergence-hardness. It is worth noting that no feasible solutions have been found in (c) and (l).

### 7.6  Analysis of Experimental Results

From the above performance comparisons on DAS-CMOP1-9 and DAS-CMaOP1-9 with a set of difficulty triplets, it is clear that each type of constraint function generated corresponding difficulties for MOEA/D-CDP, NSGA-II-CDP, C-MOEA/DD, and C-NSGA-III. With the increase of each value in the difficulty triplet, the problems became more and more difficult for MOEA/D-CDP, NSGA-II-CDP, C-MOEA/DD, and C-NSGA-III to solve.

Furthermore, it can be concluded that NSGA-II-CDP performed better than MOEA/D-CDP on most of the DAS-CMOPs with simultaneous diversity-, feasibility-, and convergence-hardness. MOEA/D-CDP performed better than NSGA-II-CDP on most of the convergence-hard DAS-CMOPs. C-NSGA-III performed better than C-MOEA/DD on most DAS-CMaOPs with feasibility-hardness. C-MOEA/DD was significantly better than C-NSGA-III on most of the convergence-hard DAS-CMaOPs. Therefore, through comprehensive and systematic experiments, it is revealed that mechanisms of MOEA/D-CDP may be more effective in solving convergence-hard CMOPs, while mechanisms of NSGA-II-CDP may be more effective in solving CMOPs with simultaneous diversity-, feasibility-, and convergence-hardness. It is also revealed that mechanisms in C-NSGA-III may be more effective in solving feasibility-hard CMaOPs, while mechanisms of C-MOEA/DD may be more effective in solving CMaOPs with convergence-hardness.

## 8  Conclusion

In this work, we proposed a construction toolkit to build difficulty adjustable and scalable CMOPs. The method used to design the construction toolkit is based on three primary constraint functions identified to correspond to three primary difficulty types of DAS-CMOPs. The method is also scalable, because both the number of objectives and the number of constraints can be conveniently extended. As an example, a set of DAS-CMOPs (DAS-CMOP1-9) and a set of DAS-CMaOPs (DAS-CMaOP1-9) were generated using this construction toolkit. To verify the usefulness of the suggested test instances, comprehensive experiments were conducted to test the performance of two popular CMOEAs (MOEA/D-CDP and NSGA-II-CDP) on DAS-CMOPs and two CMaOEAs (C-MOEA/DD and C-NSGA-III) on DAS-CMaOPs with a variety of difficulty triplets. Through analyzing the performance of the four test algorithms, it was found that the three primary types of difficulties did exist in the corresponding test problems, and that the algorithms under test showed different behaviors in their approaches to the PFs. The observation demonstrates that the proposed method of constructing the CMOPs and CMaOPs is very effective and that the resulting functions are useful in evaluating the performance of CMOEAs and CMaOEAs. In the future, we will construct problems with both continuous and discrete decision variables to study their impact on the definition of difficulty types and levels of the optimization problems. In other future work, we will investigate the change in difficulty levels with an increasing number of variables, and test the performance of push and pull search algorithm (Fan et al., 2019) on CMOPs with a large number of variables.

## Acknowledgments

This research work was supported by Key Laboratory of Intelligent Manufacturing Technology (Shantou University), Ministry of Education, by the National Natural Science Foundation of China under grant (61175073, 61300159, 61332002, 51375287), by the Natural Science Foundation of Jiangsu Province of China under grant SBK2018022017, by the China Postdoctoral Science Foundation under grant 2015M571751, by the Science and Technology Planning Project of Guangdong Province of China under grant 180917144960530, by the Project of International, as well as Hongkong, Macao & Taiwan Science and Technology Cooperation Innovation Platform in Universities in Guangdong Province (2015KGJHZ014), by the State Key Lab of Digital Manufacturing Equipment & Technology under grant DMETKF2019020, by a grant from ANR/RRC Joint Research Scheme sponsored by the Research Grants Council of the Hong Kong Special Administrative Region, China and France National Research Agency (Project No. A-CityU101/16), and by the National Defense Technology Innovation Special Zone Projects (Shantou University and National University of Defense Technology).

## References

Bader
,
J.
, and
Zitzler
,
E
. (
2011
).
HypE: An algorithm for fast hypervolume-based many-objective optimization
.
Evolutionary Computation
,
19
(
1
):
45
76
.
Ben
Hadj-Alouane
,
A.
, and
Bean
,
J. C
. (
1997
).
A genetic algorithm for the multiple-choice integer program
.
Operations Research
,
45
(
1
):
92
101
.
Beume
,
N.
,
Naujoks
,
B.
, and
Emmerich
,
M
. (
2007
).
SMS-EMOA: Multiobjective selection based on dominated hypervolume
.
European Journal of Operational Research
,
181
(
3
):
1653
1669
.
Binh
,
T. T.
, and
Korn
,
U
. (
1997
). MOBES: A multiobjective evolution strategy for constrained optimization problems. In
The Third International Conference on Genetic Algorithms (Mendel 97)
, pp.
176
182
.
Cai
,
X.
,
Li
,
Y.
,
Fan
,
Z.
, and
Zhang
,
Q
. (
2015
).
An external archive guided multiobjective evolutionary algorithm based on decomposition for combinatorial optimization
.
IEEE Transactions on Evolutionary Computation
,
19
(
4
):
508
523
.
Cai
,
Z.
, and
Wang
,
Y
. (
2006
).
A multiobjective optimization-based evolutionary algorithm for constrained optimization
.
IEEE Transactions on Evolutionary Computation
,
10
(
6
):
658
675
.
Cheng
,
R.
,
Jin
,
Y.
,
Olhofer
,
M.
, and
Sendhoff
,
B
. (
2017
).
Test problems for large-scale multiobjective and many-objective optimization
.
IEEE Transactions on Cybernetics
,
47
(
12
):
4108
4121
.
Coello
,
C. A. C
. (
2018
). Constraint-handling techniques used with evolutionary algorithms. In
Proceedings of the Genetic and Evolutionary Computation Conference Companion (GECCO)
, pp.
773
799
.
Coello
,
C. A. C.
, and
Cortés
,
N. C
. (
2005
).
Solving multiobjective optimization problems using an artificial immune system
.
Genetic Programming and Evolvable Machines
,
6
(
2
):
163
190
.
Coello
Coello
,
C. A
. (
2002
).
Theoretical and numerical constraint-handling techniques used with evolutionary algorithms: A survey of the state of the art
.
Computer Methods in Applied Mechanics and Engineering
,
191
(
11–12
):
1245
1287
.
Coit
,
D. W.
,
Smith
,
A. E.
, and
Tate
,
D. M
. (
1996
).
Adaptive penalty methods for genetic optimization of constrained combinatorial problems
.
INFORMS Journal on Computing
,
8
(
2
):
173
182
.
Corne
,
D. W.
,
Jerram
,
N. R.
,
Knowles
,
J. D.
,
Oates
,
M. J.
et al
, et al. (
2001
). PESA-II: Region-based selection in evolutionary multiobjective optimization. In
Proceedings of the Genetic and Evolutionary Computation Conference (GECCO)
, pp.
283
290
.
Deb
,
K
. (
1999
).
Multi-objective genetic algorithms: Problem difficulties and construction of test problems
.
Evolutionary Computation
,
7
(
3
):
205
230
.
Deb
,
K
. (
2000
).
An efficient constraint handling method for genetic algorithms
.
Computer Methods in Applied Mechanics and Engineering
,
186
(
2
):
311
338
.
Deb
,
K.
(
2001
).
Multi-objective optimization using evolutionary algorithms
, Vol.
16
.
New York
:
John Wiley & Sons
.
Deb
,
K.
, and
Jain
,
H
. (
2014
).
An evolutionary many-objective optimization algorithm using reference-point-based nondominated sorting approach, Part I: Solving problems with box constraints
.
IEEE Transactions on Evolutionary Computation
,
18
(
4
):
577
601
.
Deb
,
K.
,
Pratap
,
A.
,
Agarwal
,
S.
, and
Meyarivan
,
T
. (
2002
).
A fast and elitist multiobjective genetic algorithm: NSGA-II
.
IEEE Transactions on Evolutionary Computation
,
6
(
2
):
182
197
.
Deb
,
K.
,
Pratap
,
A.
, and
Meyarivan
,
T
. (
2001
). Constrained test problems for multi-objective evolutionary optimization. In
International Conference on Evolutionary Multi-Criterion Optimization
, pp.
284
298
.
Deb
,
K.
,
Thiele
,
L.
,
Laumanns
,
M.
, and
Zitzler
,
E
. (
2005
).
Scalable test problems for evolutionary multiobjective optimization
.
Berlin
:
Springer
.
Fan
,
Z.
,
Li
,
W.
,
Cai
,
X.
,
Li
,
H.
,
Wei
,
C.
,
Zhang
,
Q.
,
Deb
,
K.
, and
Goodman
,
E.
(
2019
).
Push and pull search for solving constrained multi-objective optimization problems
.
Swarm and Evolutionary Computation
,
44:665
679
.
Hoffmeister
,
F.
, and
Sprave
,
J
. (
1996
). Problem-independent handling of constraints by use of metric penalty functions. In
Proceedings of the Fifth Annual Conference on Evolutionary Programming
, pp.
289
294
.
Huang
,
F.-z.
,
Wang
,
L.
, and
He
,
Q
. (
2007
).
An effective co-evolutionary differential evolution for constrained optimization
.
Applied Mathematics and Computation
,
186
(
1
):
340
356
.
Huband
,
S.
,
Hingston
,
P.
,
Barone
,
L.
, and
While
,
L
. (
2006
).
A review of multiobjective test problems and a scalable test problem toolkit
.
IEEE Transactions on Evolutionary Computation
,
10
(
5
):
477
506
.
Ishibuchi
,
H.
,
Akedo
,
N.
, and
Nojima
,
Y
. (
2015
).
Behavior of multiobjective evolutionary algorithms on many-objective knapsack problems
.
IEEE Transactions on Evolutionary Computation
,
19
(
2
):
264
283
.
Jain
,
H.
, and
Deb
,
K
. (
2014
).
An evolutionary many-objective optimization algorithm using reference-point based nondominated sorting approach, Part II: Handling constraints and extending to an adaptive approach
.
IEEE Transactions on Evolutionary Computation
,
18
(
4
):
602
622
.
Joines
,
J. A.
, and
Houck
,
C. R
. (
1994
). On the use of non-stationary penalty functions to solve nonlinear constrained optimization problems with GA's. In
Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence
, pp.
579
584
.
Kim
,
D. G
. (
1998
). Riemann mapping based constraint handling for evolutionary search. In
Proceedings of the 1998 ACM Symposium on Applied Computing
, pp.
379
385
.
Koziel
,
S.
, and
Michalewicz
,
Z
. (
1998
). A decoder-based evolutionary algorithm for constrained parameter optimization problems. In
International Conference on Parallel Problem Solving from Nature
, pp.
231
240
.
Koziel
,
S.
, and
Michalewicz
,
Z
. (
1999
).
Evolutionary algorithms, homomorphous mappings, and constrained parameter optimization
.
Evolutionary Computation
,
7
(
1
):
19
44
.
Laumanns
,
M.
,
Thiele
,
L.
, and
Zitzler
,
E
. (
2006
).
An efficient, adaptive parameter variation scheme for metaheuristics based on the epsilon-constraint method
.
European Journal of Operational Research
,
169
(
3
):
932
942
.
Li
,
H.
, and
Zhang
,
Q
. (
2009
).
Multiobjective optimization problems with complicated Pareto sets, MOEA/D and NSGA-II
.
IEEE Transactions on Evolutionary Computation
,
13
(
2
):
284
302
.
Li
,
H.
,
Zhang
,
Q.
, and
Deng
,
J
. (
2014
). Multiobjective test problems with complicated Pareto fronts: Difficulties in degeneracy. In
2014 IEEE Congress on Evolutionary Computation (CEC)
, pp.
2156
2163
.
Li
,
K.
,
Deb
,
K.
,
Zhang
,
Q.
, and
Kwong
,
S
. (
2015
).
An evolutionary many-objective optimization algorithm based on dominance and decomposition
.
IEEE Transactions on Evolutionary Computation
,
19
(
5
):
694
716
.
Liu
,
H.-L.
,
Chen
,
L.
,
Deb
,
K.
, and
Goodman
,
E
. (
2017
).
Investigating the effect of imbalance between convergence and diversity in evolutionary multi-objective algorithms
.
IEEE Transactions on Evolutionary Computation
,
21
(
3
):
408
425
.
Liu
,
H.-L.
,
Gu
,
F.
, and
Zhang
,
Q
. (
2014
).
Decomposition of a multiobjective optimization problem into a number of simple multiobjective subproblems
.
IEEE Transactions on Evolutionary Computation
,
18
(
3
):
450
455
.
Mallipeddi
,
R.
, and
Suganthan
,
P. N
. (
2010
).
Ensemble of constraint handling techniques
.
IEEE Transactions on Evolutionary Computation
,
14
(
4
):
561
579
.
Mezura-Montes
,
E.
, and
Coello
,
C. A. C
. (
2011
).
Constraint-handling in nature-inspired numerical optimization: Past, present and future
.
Swarm and Evolutionary Computation
,
1
(
4
):
173
194
.
Michalewicz
,
Z.
, and
Nazhiyath
,
G.
(
1995
).
Genocop III: A co-evolutionary algorithm for numerical optimization problems with nonlinear constraints
. In
Proceedings of 1995 IEEE International Conference on Evolutionary Computation
, pp.
647
651
.
Osyczka
,
A.
, and
Kundu
,
S
. (
1995
).
A new method to solve generalized multicriteria optimization problems using the simple genetic algorithm
.
Structural Optimization
,
10
(
2
):
94
99
.
Paredis
,
J
. (
1994
). Co-evolutionary constraint satisfaction. In
International Conference on Parallel Problem Solving from Nature
, pp.
46
55
.
Phan
,
D. H.
, and
Suzuki
,
J
. (
2013
). R2-IBEA: R2 indicator based evolutionary algorithm for multiobjective optimization. In
2013 IEEE Congress on Evolutionary Computation
, pp.
1836
1845
.
Qu
,
B. Y.
, and
Suganthan
,
P. N
. (
2011
).
Constrained multi-objective optimization algorithm with an ensemble of constraint handling methods
.
Engineering Optimization
,
43
(
4
):
403
416
.
Ray
,
T.
,
Singh
,
H. K.
,
Isaacs
,
A.
, and
Smith
,
W
. (
2009
). Infeasibility driven evolutionary algorithm for constrained optimization. In
Constraint-Handling in Evolutionary Optimization
, pp.
145
165
.
Runarsson
,
T. P.
, and
Yao
,
X
. (
2000
).
Stochastic ranking for constrained evolutionary optimization
.
IEEE Transactions on Evolutionary Computation
,
4
(
3
):
284
294
.
Runarsson
,
T. P.
, and
Yao
,
X
. (
2005
).
Search biases in constrained evolutionary optimization
.
IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews
,
35
(
2
):
233
243
.
Salcedo-Sanz
,
S
. (
2009
).
A survey of repair methods used as constraint handling techniques in evolutionary algorithms
.
Computer Science Review
,
3
(
3
):
175
192
.
Srinvas
,
N.
, and
Deb
,
K
. (
1994
).
Multi-objective function optimization using non-dominated sorting genetic algorithms
.
Evolutionary Computation
,
2
(
3
):
221
248
.
Surry
,
P. D.
, and
Radcliffe
,
N. J.
(
1997
).
The COMOGA method: Constrained optimisation by multi-objective genetic algorithms
.
Control and Cybernetics
,
26:391
412
.
Takahama
,
T.
, and
Sakai
,
S
. (
2005
).
Constrained optimization by applying the $α$ constrained method to the nonlinear simplex method with mutations
.
IEEE Transactions on Evolutionary Computation
,
9
(
5
):
437
451
.
Takahama
,
T.
,
Sakai
,
S.
, and
Iwane
,
N
. (
2005
). Constrained optimization by the $ɛ$ constrained hybrid algorithm of particle swarm optimization and genetic algorithm. In
AI 2005: Advances in Artificial Intelligence
, pp.
389
400
.
Tanaka
,
M.
,
Watanabe
,
H.
,
Furukawa
,
Y.
, and
Tanino
,
T.
(
1995
).
GA-based decision support system for multicriteria optimization
. In
IEEE International Conference on Systems, Man and Cybernetics, Intelligent Systems for the 21st Century.
, Vol.
2
, pp.
1556
1561
.
Tessema
,
B.
, and
Yen
,
G. G
. (
2006
). A self adaptive penalty function based algorithm for constrained optimization. In
2006 IEEE Congress on Evolutionary Computation
, pp.
246
253
.
Ullah
,
A. S. B.
,
Sarker
,
R.
, and
Lokan
,
C
. (
2012
).
Handling equality constraints in evolutionary optimization
.
European Journal of Operational Research
,
221
(
3
):
480
490
.
Van Veldhuizen
,
D. A.
, and
Lamont
,
G. B
. (
1998
). Evolutionary computation and convergence to a Pareto front. In
Late Breaking Papers at the Genetic Programming Conference
, pp.
221
228
.
Wang
,
Y.
,
Cai
,
Z.
,
Zhou
,
Y.
, and
Zeng
,
W
. (
2008
).
An adaptive tradeoff model for constrained evolutionary optimization
.
IEEE Transactions on Evolutionary Computation
,
12
(
1
):
80
92
.
While
,
L.
,
Bradstreet
,
L.
, and
Barone
,
L
. (
2012
).
A fast way of calculating exact hypervolumes
.
IEEE Transactions on Evolutionary Computation
,
16
(
1
):
86
95
.
Woldesenbet
,
Y. G.
,
Yen
,
G. G.
, and
Tessema
,
B. G
. (
2009
).
Constraint handling in multiobjective evolutionary optimization
.
IEEE Transactions on Evolutionary Computation
,
13
(
3
):
514
525
.
Wolpert
,
D. H.
, and
Macready
,
W. G
. (
1997
).
No free lunch theorems for optimization
.
IEEE Transactions on Evolutionary Computation
,
1
(
1
):
67
82
.
Yuan
,
Y.
,
Xu
,
H.
,
Wang
,
B.
,
Zhang
,
B.
, and
Yao
,
X
. (
2016
).
Balancing convergence and diversity in decomposition-based many-objective optimizers
.
IEEE Transactions on Evolutionary Computation
,
20
(
2
):
180
198
.
Zhang
,
Q.
, and
Li
,
H
. (
2007
).
MOEA/D: A multiobjective evolutionary algorithm based on decomposition
.
IEEE Transactions on Evolutionary Computation
,
11
(
6
):
712
731
.
Zhang
,
Q.
,
Zhou
,
A.
,
Zhao
,
S.
,
Suganthan
,
P. N.
,
Liu
,
W.
, and
Tiwari
,
S.
(
2008
).
Multiobjective optimization test instances for the CEC-2009 special session and competition
.
Technical Report
.
University of Essex, Colchester, UK and Nanyang Technological University, Singapore, Special Session on Performance Assessment of Multi-Objective Optimization Algorithms
, pp.
1
30
.
Zitzler
,
E.
, and
Künzli
,
S
. (
2004
). Indicator-based selection in multiobjective search. In
Parallel Problem Solving from Nature
, pp.
832
842
.
Zitzler
,
E.
,
Laumanns
,
M.
,
Thiele
,
L.
,
Zitzler
,
E.
,
Zitzler
,
E.
,
Thiele
,
L.
, and
Thiele
,
L.
(
2001
).
SPEA2: Improving the strength Pareto evolutionary algorithm
.
TIK-Rep 103
, pp.
1
21
.
Swiss Federal Institute of Technology
.
Zitzler
,
E.
, and
Thiele
,
L
. (
1999
).
Multiobjective evolutionary algorithms: A comparative case study and the strength Pareto approach
.
IEEE Transactions on Evolutionary Computation
,
3
(
4
):
257
271
.
Zitzler
,
E.
,
Thiele
,
L.
,
Laumanns
,
M.
,
Fonseca
,
C. M.
, and
da Fonseca
,
V. G
. (
2003
).
Performance assessment of multiobjective optimizers: An analysis and review
.
IEEE Transactions on Evolutionary Computation
,
7
(
2
):
117
132
.