Effect of Objective Normalization and Penalty Parameter on Penalty Boundary Intersection Decomposition-Based Evolutionary Many-Objective Optimization Algorithms

In solving optimization problems having two or more objectives, most evolutionary multiobjective and many-objective optimization (EMO/EMaO) algorithms use a diversity measure by computing relative distance of population members in the objective space. In NSGA-II (Deb et al., 2002), the crowding distance operator computes a sort of Manhattan distance between two neighboring non-dominated points. In MOEA/D (Zhang et al., 2010) with the Penalty Boundary Intersection (PBI)-based approach, two orthogonal distances—one along a supplied decomposition vector and one orthogonal to it of every population member—need to be computed for the fitness-based selection operator. In NSGA-III (Deb and Jain, 2014), only the distance perpendicular to the decomposition vector is needed. If all objectives are such that all function values have a similar scale or range of values, the so-called uniformly scaled problems, like in DTLZ problems (Deb et al., 2005), no normalization operator may be needed. On the other hand, an EMO algorithm should not be developed purely based on its performance on uniformly scaled problems alone, as most practical problems involve different functionalities, such as cost, efficiency, quality, etc., which are likely to have non-uniformly scaled objective values. Thus, any distance computation between two population members is relevant and useful only if the objectives are normalized properly so that distance along each objective is given an almost equal importance.

In this article, we estimate the sensitivity of association and distances of population members to specific decomposition vectors due to the variation of estimated ideal and nadir vectors (⁠$z0$ and $z1$⁠) with generations. We define a sensitivity ratio comparing the sensitivity of MOEA/D-PBI algorithm over NSGA-III and compute its value theoretically as a function of PBI's penalty parameter $θ$⁠. We also find a theoretical connection between the lower bound of $θ$ and the tangent of the PF. Theoretically, these two aspects, sensitivity due to instability in normalization and geometric bound from PF, dictate the most preferred value of $θ$⁠. We then apply two PBI-based EMO algorithms to a series of DTLZ and WFG problems and demonstrate the validity of our theoretical results.

In the remainder of the article, we provide a brief overview of the preliminaries of a multiobjective optimization problem. In Section 2, we describe the basic principles of decomposition-based EMO algorithms. The PBI-metric--based selection approach is described. We then present a theoretical analysis of a sensitivity ratio of the PBI-metric to the orthogonal distance metric-based fitness selection approaches on instabilities of the normalization procedure described in Section 3. Section 4 presents extensive simulations of PBI-based NSGA-III and MOEA/D algorithms to DTLZ and WFG problems having 3 to 15 objectives to validate the theoretical results obtained in the previous section. The experimental results on convex and non-convex PFs are presented and analyzed. Finally, conclusions are drawn in Section 5.

Decomposition-based EMO algorithms are becoming more popular in dealing with multiobjective optimization problems (MOPs), especially MOPs with more than three objectives (Deb et al., 2007; Khare et al., 2003). Decomposition-based EMOs need a set of reference vectors to guide the population search, and those vectors are either used for objective aggregation (Ishibuchi and Murata, 1998; Zhang et al., 2010; Li and Zhang, 2009) or diversity and convergence enhancement (Liu et al., 2013, 2018; Deb and Jain, 2014; Li et al., 2015; Yuan et al., 2016; Cheng et al., 2016). For example, the Multiobjective Evolutionary Algorithm Based on Decomposition (MOEA/D) (Zhang and Li, 2007; Zhang et al., 2010; Li and Zhang, 2009) decomposes an MaOP into a number of scalar optimization sub-problems by reference vectors. MOEA/D-M2M (Liu et al., 2013) is a new variant of MOEA/D for population decomposition, and it decomposes an MaOP into a number of many-objective optimization subproblems by direction vectors. Deb and Jain (2014) proposed the third generation non-dominated sorting genetic algorithm (NSGA-III) by using reference directions to enhance the convergence and maintain the diversity. Reference points are also used for decomposition in MOEA/DD (Li et al., 2015) and $θ$-DEA (Yuan et al., 2016). Cheng et al. (2016) proposed the Reference Vector-guided Evolutionary Algorithm (RVEA). It is worth noting that we collectively call these vectors as decomposition vectors in this article, as they are essentially used to decompose the overall problem into several interacting subproblems.

There are two steps in NSGA-III's selection. The first step is non-dominated sorting, where the parents and offspring are combined together and are sorted in different non-domination levels. From the first non-domination level, solutions are kept until the size of selected solutions exceeds the initial population size. The second step is a niching selection, in which the remaining solutions are selected from the last accepted level by a niche-preservation operator. The resulting solutions of the first step selection are associated with the evenly distributed decomposition vectors (called reference points in the original NSGA-III study) according to $d2(F'(x))$⁠. During the niching selection, the resulting solutions except for those in the last acceptable level are first selected. After that, solutions in the last acceptable level associated with a less crowded reference line are preferred to be selected, and this procedure is called $d2$-based selection in this study.

To have an idea of the relative importance of these two vectors, first, we observe how the estimated ideal point $z0$ is varied over generations in a standard EMO run, NSGA-III (Deb and Jain, 2014), on test problems. Figure 1 shows how the norm of the observed ideal point (⁠$∥z0∥$⁠) varies with the generation counter for three- to 15-objective instances of DTLZ1. The norm value is expected to be zero for these problems.

The median of 15 runs is plotted. It is clear that the ideal point estimation gets settled very early (not necessarily the original point) during a simulation. A similar observation is also made for other problems of this study and has also been observed in other studies in the past (Seada et al., 2018; Bechikh et al., 2010). For brevity, those plots are not shown here. These plots show that, except for a few early generations, the ideal point $z0$ does not contribute much to the normalization procedure in most generations.

It is clear from Equation 4 that any variation in the estimation of $α$ from one generation to another will cause a variation in the computation of the normalized objective vector, $fn$⁠. This will then cause a variation in the calculation of distance metrics $d1$ and $d2$ for the purpose of PBI or NSGA-III fitness assignment procedure. Although we keep Das and Dennis's (1998) vectors invariant in an EMO simulation, the generation-wise change in $α$ destabilizes the effective normalized objective values.

Now we are ready to define a sensitivity ratio between PBI-based fitness and orthogonal distance-based fitness values due to the perturbation of the revised decomposition vector $v$⁠. We define the sensitivity ratio as the absolute ratio of relative perturbation of PBI-metric to orthogonal distance metric by infinitesimal analysis, as follows:

Thus, in such cases, the percentage sensitivity of the PBI distance metric is smaller than the percentage sensitivity of the $d2$ distance metric. Also, $ρ$ is small for a small $θ$ value, meaning that PBI-based approaches are less sensitive compared to $d2$-based approach for small $θ$⁠. It also implies that if NSGA-III's orthogonal distance metric (⁠$d2$⁠) is replaced with the PBI-metric (⁠$d1+θd2$⁠, with the introduction of a parameter $θ>0$⁠), it should introduce a smaller effect in the selection operator due to the instabilities in the normalization process.

We now investigate the validity of the condition $θ≥(β2-1)/(2β)$ by noting that the ratio $β=d2/d1$ is related to the specified decomposition (or reference) vectors. Figure 3 shows the geometric meaning of the two distances in a two-objective case.

When an equi-angled reference direction $(1,1,…,1)T/M$ to each objective axis is used, the controlled NSGA-III performs the best on both three and five-objective instances of DTLZ2 with the smallest $θ$ used in the study. The noise coming from the normalization of objectives makes a smaller effect when reducing $θ$ and the resulting algorithm works better. Please note that the Wilcoxon test has shown that the best one is significantly better than the others.

However, the latter part of this table indicates that when a skewed reference direction (closer to the $fM$ objective axis: $(a,a,…,a,0.9)T$ with $a=0.19/(M-1)$⁠) is used, there exists an optimal $θ$ value which does not correspond to the smallest $θ$ considered. That is, the performance gets better with smaller $θ$⁠, but beyond certain $θ$⁠, any further reduction causes the performance to deteriorate. We explain this behavior of the controlled NSGA-III in the following paragraph.

Consider Figure 5 for a two-objective minimization problem having a non-convex PF.

An interesting phenomenon happens for the point $C$⁠, for which any positive $θ$ would make point $A$ better than point $C$⁠. Thus, if the reference direction passes through the intermediate part of the PF, it is expected to have points around the reference direction (such as $B$ and $C$⁠), and although points like $B$ will not allow point $A$ to be chosen for a small $θ$⁠, but points like $C$ will make it possible to converge to $A$ for any $θ$⁠. This is the reason, for which for non-convex PF scenarios (such as DTLZ2), an equi-angled reference direction is able to find near-$A$ like point with a very small $θ$⁠, such as 0.1 (Table 1). Our sensitivity analysis predicted that a smaller $θ$ handles the normalization sensitivity better. Thus, based on the sensitivity analysis performed and the above geometric argument in favor of small $θ$ with intermediate reference lines, NSGA-III with the PBI-metric having a very small $θ$ produced the best result.

However, as the reference lines (⁠$w(2)$⁠) closer to the axis directions are considered, the gradient concept still determines the angle $α$⁠, such as $α'$ in Figure 5, leading to a minimum $θmin'=cotα'$ for targeted Pareto-optimal points like $D$ to be better than point $E$⁠. Of course, unlike points like $F$⁠, point $D$ will be judged better for any positive $θ$⁠, but since points like F lie on the extreme side of the reference line, there may not be many points present there in the population. Therefore, an EMO algorithm will require a minimum $θ$ (⁠$=θmin'$ in this case) to select a point close to D. This is why in Table 1, we observed a relatively larger $θ$ to work the best.

The above discussions make the following aspects clear:

1. The normalization uncertainty (due to instability in ideal and nadir point fixation), a smaller $θ$ value in the PBI-metric introduces smaller sensitivity. From this perspective, the use of the smallest $θ$ (⁠$>0$⁠) is the best strategy.

2. However, for non-convex PFs, the angle made by the tangent of the PF at the Pareto-optimal point and the corresponding reference direction determines a minimum required $θ$ for a PBI-metric--based EMO algorithm to converge near the true Pareto-optimal point from the commonly available neighboring points. For intermediate reference directions, commonly available points are usually all around, hence there is no minimal lower bound on $θ$ for points to find a way to reach the targeted Pareto-optimal point. But for near-boundary reference directions, commonly available points are not always available all around, and a finite minimal $θ(≥cotα)$ is needed for finding the targeted Pareto-optimal point.

We shall discuss the effect of the geometric lower bound on convex PFs later, but before we do that, we present results of NSGA-III and MOEA/D with PBI-metric on standard DTLZ and WFG problems, which have non-convex PFs, in support of our above arguments.

From our theoretical results on normalization sensitivity, it is expected that with a decrease in $θ$ value, the performance of both algorithms on non-convex problems should get better, but to obtain the extreme and boundary solutions, a lower bound on $θ$ is expected. In this section, extensive simulations are conducted to validate this theory and identify best parameter values of $θ$⁠, when the PBI-metric is introduced to NSGA-III and MOEA/D.

A number of scalable MaOP test problems from the DTLZ family (Deb et al., 2005) and WFG family (Huband et al., 2006) are used for this purpose. Problems DTLZ1-DTLZ4, and WFG5-WFG8 with the number of objectives $M=3$⁠, 5, 8, 10, and 15 are tested in this study. The number of decision variables is set as $n=M+4$ for DTLZ1, $n=M+9$ for DTLZ2-4, and $n=M+19$ for WFG5-8. The IGD metric (Bosman and Thierens, 2003) is used to measure the performance of the two algorithms. The intersecting points of decomposition vectors and PF are used as the reference points for IGD calculation (Li et al., 2015), and thus the number of reference points is the same as that of decomposition vectors.

The general settings of NSGA-III, MOEA/D and their modifications are as follows:

• Initialization of the reference points are kept the same as the original NSGA-III paper (Deb and Jain, 2014). Population size $N$⁠, divisions $H$⁠, and respective reduction factor for layer-wise Das and Dennis (1998) reference vectors are shown in Table 2.

• The number of generations for DTLZ1-4, and WFG5-8 are shown in Table 3.

• The SBX operator with $pc=1$ and $ηc=30$⁠, and polynomial mutation with $pm=1/n$ and $ηm=20$ are used in all experiments.

• For MOEA/D, $T=20$⁠, $δ=0.9$⁠, and $nr=2$⁠, are used according to suggestions of Zhang et al. (2010).

First, we modify the selection procedure of the original NSGA-III by replacing the $d2$-based selection with the PBI-metric. Note that the original NSGA-III can be considered to have used the PBI-metric with $θ=∞$⁠. For the Modified-1 of NSGA-III, we use $θ=100$⁠, for Modified-2, we use $θ=5$⁠, and for Modified-3, we use $θ=1$⁠. For points associated with $M$ axis directions, we use a large $θ=106$ such that the orthogonal distance to the axis line is used as the fitness measure. In Table 4, we have observed that this stabilizes the overall normalization procedure and without this modification, the PBI-based NSGA-IIIs do not converge well.

Table 4 shows that a very small $θ$ (⁠$=0.1$⁠) does not produce best results for most DTLZ problems. For DTLZ1 problem, $θ=5$ and for other DTLZ problems $θ=1$ perform the best. Clearly, a very high $θ$ is good for geometric consideration, but it is not a good choice from the sensitivity point of view. The documented good $θ$ values around 1--5 in the literature agree well with our controlled experimental study in Table 1.

Next, we apply the PBI-based NSGA-IIIs on the WFG family in Table 5. Here, for most problems $θ=1$ works better.

Both tables reveal an interesting result, which the original NSGA-III study (Deb and Jain, 2014) did not show. If the original NSGA-III's orthogonal distance metric is replaced with the PBI-metric having $θ≈1$⁠, the performance is better. The previous MOEA/DD (Li et al., 2015) and $θ$-DEA (Yuan et al., 2016) have reported similar observations. This stays as an important result for NSGA-III researchers and users.

Here, we provide results on the original MOEA/D and MOEA/D modifications on the DTLZ and WFG problems. In the original MOEA/D, PBI selection with $θ=5$ was suggested. In this study, we include three modifications: Modified-1 with $θ=1$⁠, Modified-2 with $θ=10$⁠, and Modified-3 with $θ=100$⁠. Tables 6 and 7 present the IGD metric values on DTLZ and WFG problems, respectively.

Clearly, $θ=5$ works the best for both families of problems. Again, larger $θ$ values do not produce good results on WFG problems and too small (⁠$θ=1$⁠, which worked very well for NSGA-III procedure) does not work well on both families of problems with MOEA/D. Similar results were reported in another study (Ishibuchi et al., 2016a). We now describe the reason for MOEA/D requiring a larger $θ$ compared with PBI-based NSGA-III in the following paragraph.

Consider Figure 5 again. In NSGA-III, there is no neighborhood concept and a point close to a reference line is declared to be associated with the line. For a given $θ$⁠, an ND point closer to a reference line will be chosen by the PBI-metric. On the other hand, in MOEA/D, points within a prespecified neighborhood size (usually $T=20$⁠) are used for the PBI-metric computation. Thus, in MOEA/D, points are expected to be relatively far away from a reference line used for their PBI-metric computations, compared with the same in NSGA-III. For the reference line $w(1)$⁠, if point $B$ is associated with it, for MOEA/D it can be point K which is in the neighborhood of the reference line. In comparison with a point $B$⁠, which is closer to the targeted Pareto-optimal point $A$⁠, point K will be judged better than $B$ for a large $α$ (or, small $θ$⁠). A relatively large $θ$ (about $θ=2$⁠, meaning $α=cot-1(2)=$ 26.57 deg) is needed to establish that point $B$ is better than point K. However, if points like K are not allowed to be associated with the reference line $w(1)$⁠, the above situation does not arise and a small $θ$⁠, found adequate in our sensitivity analysis would produce a better normalization accuracy. While other researchers (Ishibuchi et al., 2015, 2016b) have made similar arguments for relatively large $θ$ requirement with MOEA/D, our argument comes from two separate opposing requirements: smaller $θ$ is better from sensitivity due to normalization uncertainty and large $θ$ for accepting closer to reference line points.