Maximum Diversity Problem . A Multi-Objective Approach

The Maximum Diversity (MD) problem is the process of selecting a subset of elements where the diversity among selected elements is maximized. Several diversity measures were already studied in the literature, optimizing the problem considered in a pure mono-objective approach. This work presents for the first time multi-objective approaches for the MD problem, considering the simultaneous optimization of the following five diversity measures: (i) Max-Sum, (ii) Max-Min, (iii) Max-MinSum, (iv) Min-Diff and (v) Min-P-center. Two different optimization models are proposed: (i) Multi-Objective Maximum Diversity (MMD) model, where the number of elements to be selected is defined a-priori, and (ii) Multi-Objective Maximum Average Diversity (MMAD) model, where the number of elements to be selected is also a decision variable. To solve the formulated problems, a Multi-Objective Evolutionary Algorithm (MOEA) is presented. Experimental results demonstrate that the proposed MOEA found good quality solutions, i.e. between 98.85% and 100% of the optimal Pareto front when considering the hypervolume for comparison purposes.


Introduction
In multiple contexts, the process of selecting objects, ideas, people, projects or resources is an activity frequently performed by individuals, companies or governments and in many cases it is required that the selected elements have different characteristics, so they can represent diversity.The quality of the selection is based on one or more criteria: economical, spatial, affective, political, among others.For example, people must select the methods of transportation and the routes they will use to reach their destination according to price, travel time and comfort.At the end of the month, a worker will have to select which goods or services are the right ones to spend his salary on, in order to satisfy his needs.Another case is vacation planning; a person will have to choose between a given number of destinations to visit according to price and the pleasure they will provide (1).An investor will seek to distribute his or her capital among different investments in such a way as to diversify the risk but to maximize the expected return, and therefore will try not to have much money invested in similar activities that may be affected by an economical crisis that impacts operations across a sector (2).
The growing interest in dealing with diversity in recent years has motivated efforts to study the management of equity, i.e., organizations are becoming increasingly interested in ensuring an equitable treatment of individuals or institutions (3).In fact the Operational Research area has studied several cases, which are summarized as follows: In logistics, a key problem is locating logistic units that are mutually competitive, such as warehouses, plants or cargo transportation centers, so that their activities are not redundant between logistic units (4).An additional problem is the selection of sites in a number of different locations for facilities that represent some level of danger to the population, such as weapons and explosives stores, prisons or garbage dumps, while also providing an optimal location in terms of location (5).
Another case could be the problem of jury panel composition, which is the list of people from which jury members can be selected for a particular trial (6).Efficiently searching information on the Internet could be another relevant problem, so that the result should be a group of sites with diverse information (7).
In the design of new drugs, through the process known as High Throughput Screening, companies have a large library of molecules (in the millions) that they use repeatedly for each new project in order to identify successes, while the selection of a diverse subset would ensure that the cost of this process does not become excessive.In this context, Meinl proposed in (8) some diversity measures for this case.
The problem of selecting a work team to undertake a project, could also be considered as a diversity problem, where it is usually desirable to choose members according to their aptitudes, abilities to work in teams, ages and other factors that seem relevant.In the analysis of a social group, one might be interested in studying its diversity by means of a detailed study of some representative individuals; these selected individuals should have diverse characteristics which must be observed across the whole group, while also being the best for the objective of the team.
Recent studies, such as that of Castillo et al. (9), have determined that diversity in a group of people increases the ability of these groups to solve problems, and therefore contributes to obtaining more efficient working groups in companies, schools, government, etc.These authors provide a theoretical justification for this empirically known phenomenon.Pioneering researchers in this area such as Page (10) state that: Various perspectives and tools allow groups of people to find more and better solutions and contribute to total productivity.As a result, the problem of identifying diverse groups of people becomes a key point in large companies and other institutions.
In all the previously mentioned cases, it is a question of choosing the best subset of elements from a large set of possibilities.In many cases it is desirable to establish that the selected elements are not alike, but rather retain sufficiently different characteristics to represent the diversity existing in the original set.
Maximizing the diversity selection, i.e. finding the most diverse subset is known to be an NP-hard decision problem (11).This problem is commonly known as a Maximum Diversity (MD) problem (12).It is important to consider that simple procedures that seemingly offer efficient solutions lead to poor decisions; only efficient mathematical models can guarantee efficient solutions.
The MD problem presents two main research challenges: • first, there are many existing distance measures to evaluate different elements; • second, there are many existing diversity measures to evaluate different subsets of elements.
This work focuses on the second presented challenge studying the most relevant diversity measures in the literature.
The most studied measure is the Max-Sum Diversity Measure (13), in which the sum of distances between the selected elements is maximized.Heuristics (14) and meta-heuristics (15) have been proposed considering this diversity measure.
The Max-Min Diversity Measure, in which the minimum distance between the selected elements is maximized, has been well documented in recent studies (16).The specialized literature also includes the Max-MinSum Diversity Measure (17), in which the smallest total distance associated with each selected element is maximized.In the Minimum Differential (Min-Diff) Diversity Measure (17), the difference between the maximum sum and the minimum sum of the distances to the other selected elements is minimized.The Min-P-center Diversity Measure (2), in which the greatest of the minimum distances is minimized was also presented.
Prokopyev et al. introduced in (17) the Max-Mean Diversity Measure that consists in maximizing the average distance between the selected elements.Martí et.al. (18) proposed a GRASP algorithm with a Path Relinking in which the local search is based on the Variable Neighborhood methodology for this diversity measure.
If only one diversity measure is optimized, a classical mono-objective approach is considered.However, different diversity measures may be studied as conflicting objective functions; consequently, it could be desirable to study the MD problem considering a multi-objective optimization approach, simultaneously optimizing the following diversity measures: (i) Max-Sum, (ii) Max-Min, (iii) Max-MinSum, (iv) Min-Diff and (v) Min P-Center Diversity Measure.
The main contributions of this work are two multi-objective optimization models: 1. Multi-Objective Maximum Diversity (MMD) model, where the number of elements to be selected is defined a-priori.
2. Multi-Objective Maximum Average Diversity (MMAD) model, where the number of elements to be selected is a decision variable and the target is to optimize the average of each diversity model above presented.
The remainder of this work is structured as follows: Section II describes preliminary concepts while Section III presents multi-objective optimization problems.Section IV describes the proposed multi-objective problem formulation while Section V presents the proposed algorithm.Section VI summarizes experimental results while conclusions and future works are left to Section VII.

Preliminary Concepts
The following sub-sections present relevant concepts related to the main contributions of this work.
• The set of elements from which a subset should be chosen is denoted as N = {n i } and its cardinality is denoted as |N |.
• A selected subset is denoted as M = {m i }, and its cardinality is denoted as |M |.
• The distance between each element n i to every element n j is denoted as d i,j .
In the Operations Research literature, the classical mono-objective MD problem can be formulated as a maximization problem: where: div(M ) is a chosen diversity measure that could be any of the five alternative measures above mentioned, to be presented in the following sub-sections.These diversity measures require different ways of measuring distance between elements, according to the specific context as described next.

Distance Measure 1: Euclidean Distance
To measure the diversity of a subset M given by a diversity measure div(M ), it is required to first have a clear definition of the connection, distance, or dissimilarity between each pair (m i , m j ) ∈ M .The estimation of this distance depends on the particular problem that is being studied.In complex systems like social groups, a fundamental operation is the assessment of the dissimilarity between each individual pair.In the majority of applications, each element is supposed to be representable by a collection of attributes (1).Let x i,k be the k th attribute of element m i , where k = 1, ..., K. Then the distance between elements m i and m j may be defined as: In this case d i,j is the Euclidean distance (3) between m i and m j , in which larger values indicate greater dissimilarity, but other distance measures can also be considered.

Distance Measure 2: Cosine Distance
Another common distance measure in the context of dissimilarity is the cosine distance, formulated as: The cosine distance between two elements can be viewed as the angle between their attribute vectors, where a large angle between elements indicates a large degree of dissimilarity.It takes values in [-1, 1] reflecting the affinity between the individuals, as presented in (3).
Several distance measures were already studied in the literature.Sung-Hyuk Cha presents in (19) 45 distance measures definitions.However, this work will not consider the origin of the considered distance, it will simply take for granted that a distance measure between each pair of elements exist.
Once the problem of estimating the distance between pairs of elements is determined, the following step is establishing the problem of measuring diversity.

Diversity Measure 1: Max-Sum
The diversity of a given subset is represented as the maximum sum of the distances between the selected elements (18).The Max-Sum diversity measure can be formulated as: This measure is used in the problem of locating logistics units that are mutually competitive, such as warehouses, plants or cargo transportation centers, in such a way that their activities are not redundant between different logistic units (4).It is also used in the problem of jury panel composition, which is the list of people from which jury members can be selected for a particular trial (6).

Diversity Measure 2: Max-Min
The diversity of a given subset is represented as the minimum distance between the selected elements, that have to be maximized (18).The Max-Min diversity measure can be formulated as: This measure is used for example in the location of dangerous, contaminating, undesirable, or strategic installations, where it is important to avoid the vulnerability, hazards or accidents, caused by their proximity to each other.In those examples, the diversity measure that is defined is that of the shortest distance that exists between the selected elements (5).

Diversity Measure 3: Max-MinSum
It consists in selecting a set M ⊂ N of |M | elements such that the smallest total distance associated with each selected element m i is maximized (3).The Max-MinSum diversity measure can be formulated as: This measure is used in equity problems, for example in the context of facility location problems, where the fairness among candidate facility locations is as relevant as the diversity of the selected locations.These kind of problems also have applications in the context of urban public facility location or the selection of homogeneous groups (20).

Diversity Measure 4: Minimum Differential (Min-Diff )
It consists in minimizing the difference between the maximum sum and the minimum sum of the distances to the other selected elements.It is known to be strongly NP-hard, and it remains NP-hard even if sign restrictions for distances between elements are imposed (18).The Min-Diff diversity measure can be formulated as: This measure like the Max-MinSum diversity measure is used in equity problems and it has the same applications (20).

Diversity Measure 5: Min-P-Center
In contrast to the measurements mentioned above, the Min-P-center measure is not calculated with the selected items, but requires the whole original set.Note that in this case it is important to minimize the largest of these minimum distances (8).The Min-P-center diversity measure is formulated as: The evaluation of Equation ( 8) requires O(m × n) operations, the assessment of possible diversity is more costly than in previous cases.This measure is useful, for example, to (8): • locate a school so that the most distant student, walks as little as possible; • locate ambulances so that the most distant patient is treated as soon as possible.

Proposed Diversity Model 1: Multi-Objective Maximum Diversity (MMD) Model
The proposed MMD model consists in simultaneously optimizing the previously mentioned diversity measures: (i) Max-Sum, (ii) Max-Min, (iii) Max-MinSum, (iv) Min-Diff and (v) Min-P-center, taking for granted that a distance measure between each pair of element is given and the number of elements to be selected is defined a-priori.This proposed MMD model is later formally defined in Equation ( 12).
2.9 Proposed Diversity Model 2: Multi-Objective Maximum Average Diversity (MMAD) Model In the case that the number of selected elements |M | is unknown, all considered diversity measures mentioned before can be also used to find convenient numbers of selected elements |M |, by obtaining the average of each diversity measure.The proposed MMAD model can be formulated as: where: q may identify any of the above mentioned diversity measures, any combination of them or even all of them.
In the literature, the only diversity measure that also optimizes |M | is the Max-Sum (div 1 (M )), which is also known as Max-Mean diversity measure (18).This work also optimizes the number |M | of selected elements like the Max-Mean for the other diversity measures (Max-Min, Max-MinSum, Min-Diff and Min-P-center).To the best of the authors' knowledge, this has not previously been proposed in the literature.This proposed diversity model is formally defined in Equation ( 13).

Multi-Objective Optimization Problems
A general Multi-Objective Optimization Problem (MOP) includes a set of p decision variables, r objective functions, and s restrictions.Objective functions and restrictions are functions of decision variables.This can be expressed as (21): Subject to : e(x) = (e 1 (x), e 2 (x), ..., e s (x)) ≥ 0, where : X denotes the decision space while the objective space is denoted by Y .Depending on the problem, optimize could mean minimize or maximize.The set of restrictions e(x) ≥ 0 determines the set of feasible solutions X f and its corresponding set of objective vectors Y f .The problem consists in finding x that optimizes f (x).
In general, there is no unique best solution but a set of solutions, none of which can be considered better than the others when all objectives are considered at the same time.This derives from the fact that there can be conflicting objectives.Thus, a new concept of optimality should be established for MOPs.
Given two decision vectors u, v ∈ X f in a pure minimization context: Thus, the following relations are possible between u and v: • u ∼ v −→ u and v are not comparable.Neither one dominates the other.
A set of all non-dominated solutions is considered as the optimal Pareto set and it is denoted as: P true = {x ∈ X f | x is not dominated by other x ∈ X f }.The corresponding image of the optimal Pareto set is known as the optimal Pareto front and it is denoted as: https://doi.org/10.19153/cleiej.21.2.2

Proposed Multi-Objective Maximum Diversity Problem Formulation
This section firstly presents the proposed formulation for the Multi-Objective Maximum Diversity (MMD) model.This model considers the simultaneous optimization of the following five objective functions: (i) Max-Sum, (ii) Max-Min, (iii) Max-MinSum, (iv) Min-Diff and (v) Min P-Center diversity measures, that can be generalized as: where: div 1 (M ), div 2 (M ) and div 3 (M ) are maximized while div 4 (M ) and div 5 (M ) are minimized.
In Equation (12), each objective function div q (M ), where q = 1, ..., 5 represents the diversity measure corresponding to the definition, previously studied in Section II.
In addition, the proposed formulation for the Multi-Objective Maximum Average Diversity (MMAD) model considers the average optimization of the five objective functions presented in the MMD model, that can be generalized as: where: again div 1 (M ), div 2 (M ) and div 3 (M ) are maximized while div 4 (M ) and div 5 (M ) are minimized.
In Equation (13), it is seen that the number |M | of selected elements is also a decision variable that is optimized.In both MMD and MMAD models, several other diversity measures can be easily included, but in this work, only the five presented diversity measures are considered as objective functions.

Example 1:
The following example is presented to better understand the proposed problem formulation.A small problem instance is studied, considering |N | = 5, |M | = 3 and the distance matrix shown in Table 1.
For this example, there are 5 3 = 10 possible solutions, but for simplicity the five considered diversity measures are only calculated for the following two solutions: Diversity Measure 1: Max-Sum Considering the above mentioned possible solutions M 1 and M 2 , the best solution taking into account the Max-Min would be the maximum, which is again M 2 , composed by [1,2,4], with 3.16 of diversity.
Diversity Measure 3: Max-MinSum where: - 28 where: - Considering the above mentioned possible solutions M 1 and M 2 , the best solution taking into account the Max-MinSum diversity measure is again M 2 , composed by [1, 2, 4], with 7.28 of diversity.

Example 2:
To demonstrate that the considered diversity measures may result in conflicting objective functions when optimizing the MD problem, and consequently some solutions may be better when considering some diversity measures, but not for others.A second example is presented considering |N | = 10, |M | = 3 and the distance matrix shown in Table 2.For this example, there are 10 3 = 120 possible solutions, but for simplicity the five considered diversity measures are only calculated for the following two solutions: Calculated results for the two considered solutions taking into account the five diversity measures are summarized as follows: In this particular case, both have the same result for div 2 , however in two out of five considered diversity measures (div 1 (M 1 ) and div 5 (M 1 ), M 1 is better than M 2 , while M 2 is better than M 1 in two other objective functions (div 3 (M 2 ) and div 4 (M 2 ).It can be seen in this example that M 1 and M 2 are not comparable (M 1 ∼ M 2 ) or equivalently neither M 1 dominates M 2 nor M 2 dominates M 1 (22).

Proposed Multi-Objective Evolutionary Algorithm
It has been demonstrated that evolutionary algorithms in general are an efficient way of solving optimization problems.A suitable representation of solutions is needed, as well as an objective function, which is subsequently optimized.This algorithm encodes potential solutions as chromosomes and uses genetic operators such as crossover and mutation as well as fitness functions (for each single objective) (23).This work proposes an efficient Multi-Objective Evolutionary Algorithm (MOEA) inspired in the Non-dominated Sorting Genetic Algorithm II (NSGA-II), which according to the specialized literature, is currently a reference algorithm even used for comparison with new multi-objective methods (24).

NSGA-II-based Algorithm
The NSGA-II algorithm proposed by Deb et al. ( 25) is presented in Algorithm 1 and it has the characteristics described below.As input data, it receives a data file containing: Algorithm 1: Considered MOEA for the proposed MMD and MMAD models.In the algorithm, a solution for the MMD model is represented by an array of |M | elements.However, for the MMAD model, in which the number |M | has to be optimized, the array has size ceil(|N |/2) and if the number of selected elements is less than the size of the array, the remaining elements are completed with the value zero.The algorithm defines some parameters such as: Population Size and Maximum Number of Evaluations for the Stopping Criterion.
In the NSGA-II-based algorithm, first a set of solutions is generated randomly for the population.Then, a solution repair is performed in case the generated solutions are unfeasible because a solution can not contain repeated elements.The repair of unfeasible solutions can be carried out in two steps: first, a feasibility verification process is performed, where the duplicate elements are detected.Then, the process of repairing unfeasible solutions is performed, where the solutions that do not follow the feasibility criteria are repaired, eliminating duplicates and generating random elements not repeated in the analyzed solutions.
As long as the stopping criterion is not reached, the following steps are performed: First, a selection operation is performed.The binary tournament selection is used, which involves "tournaments" among pairs of solutions chosen at random from the population.The winner of each tournament (the one with the best fitness) is selected for crossover (26).Then, a crossover operation is performed.The Simulated Binary Crossover (SBX) is used, which also takes two solutions and performs a crossover, returning as a result the two obtained children (26).Next, a mutation operation is performed.The Polynomial Mutation is used, which is typically applied to single solutions, modifying them accordingly; this returns the mutated solution (26).All considered genetic operators and configuration parameters for the algorithm are stardard values of the JMETAL framework (27) and also correspond to the test problems considered in (25).
After the crossover and mutation operation, the repair solutions function is performed again to repair unfeasible solutions.Finally, a non-dominated sorting is performed on each front.These results in the first front being completely non-dominant set in the current population and the second front being dominated by the solutions in the first front only and so on.Each solution in each front is assigned rank (fitness) values.Solutions in the first front are given a fitness value of 1 and solutions in the second front are assigned a fitness value of 2 and so on.The last accepted front is sorted according to a crowded-comparison operator, which is calculated for each solution and guides the selection process at the various stages of the algorithm toward a uniformly spread-out Pareto Optimal Front and only the best solutions are selected.The proposed algorithm stops after a stopping criterion (as a maximum number of evaluations is performed).

Experimental Evaluation
The following sub-sections summarize the experimental environment as well as the main findings identified in the experiments performed to validate the optimization scheme for the MMD and MMAD models.With the presented experiments, the following issues are demonstrated: • the conflicting correlation between objective functions; • the quality of solutions obtained by the proposed algorithm; and • the scalability and the evolutionary improvement of solutions obtained by the proposed algorithm.
In order to study these issues, an Exhaustive Search Algorithm (ESA) was implemented to compare the quality of solutions obtained by the NSGA-II-based Algorithm to an optimal solution set, considering the Hypervolume performance metric, which is the most accepted comparison metric in the literature (28).

Experimental Environment
Experiments were performed on an Intel(R) Pentium(R) CPU B960 @ 2.20GHz with 4GB of RAM and Windows 10 Pro 64 bits OS.The proposed MMD and MMAD models were implemented with the JMETAL framework (27), requiring Java 8.The Hypervolume metric was calculated with a Python Project (29).
Problem instances available in the website of the OPTSICOM project (30) were considered for the evaluations.Specifically, it has been used the instances in the folder GKD, which have distance matrices whose values were calculated as Euclidean distances from randomly generated points with coordinates in [0-10].

Experimental Results
The main goal of the presented experimental evaluation is to get a set of solutions that simultaneously optimize the five diversity measures studied: (i) Max-Sum, (ii) Max-Min, (iii) Max-MinSum, (iv) Min-Diff and (v) Min-P-Center.The presented NSGA-II-based algorithm was executed with the following empiricallychosen parameters: • population size: 500; • maximum number of evaluations: 250,000.

Experiment 1: Objective Function Correlation
Correlation between diversity measures can be seen in Table 3, obtained considering the result of the NSGA-II-based algorithm for a problem instance with |N | = 50 elements from which |M | = 5 had to be selected.This analysis coincides with the correlations presented by Sandoya in (3).
Correlation between Max-Sum and Max-Min is 0.64, which can be considered relatively low.Therefore, this indicates that it is not expected that good solutions for one diversity measure will be necessarily good solutions for the other.There is no significant relation between the Min-Diff and the Min-P-Center solutions and the rest of the diversity measures.Their correlations are very low in the best cases.So, these problems can be considered different enough to the rest of the diversity measures.The largest correlation coefficient is obtained between Max-Sum and Max-MinSum.Although these correlation values seem large enough, empirically it can be found that one diversity measure does not necessarily perform well on another or viceversa.Considering these correlations, it is clear that the objective functions are in conflict, thus the proposed multi-objective approach is useful for getting trade-off solutions for the MD problem.It is important to mention that as the number of evaluations and the population size increase, better solutions are usually provided by the proposed evolutionary algorithm (see Section 6.2.3).

Experiment 2: Quality of Solutions
To check the quality of solutions found by the proposed NSGA-II-based algorithm, an Exhaustive Search Algorithm (ESA) was also executed on five different instances randomly chosen from the OPTSICOM (30) website.Experimental results are presented in Tables 4 and 5.For cases where it was not possible to get solutions with the ESA, the acronym N/A is used, meaning that evaluations were not available.
Both Tables 4 and 5  specified; calculated by counting how many solutions are among the optimal solutions and how many ones are not.It can be seen that not all the solutions are non-dominated, this occurs because the NSGA-II-based shows all the non-dominated solutions found at the moment it stops.
Percentage of Hypervolume Difference (PercHypDif ): This column is calculated using the Exhaustive Search Hypervolume (HV ES ) and the NSGA-II-based Hypervolume (HV N SGA−II ) as follows: For the MMD model, the execution of the ESA was not possible for values of |N | = 100, 125, 150 and 500.For the other instances (i.e.|N | = 10 and 15), it can be seen that the ESA reaches all the solutions in a lower execution time than the proposed NSGA-II-based algorithm but in instances of |N | = 25, 30 or 50, it can be seen that the NSGA-II-based algorithm found optimal solutions in a considerable lower time.In both Tables 4 and 5, it can be seen that the proposed algorithm was able to find near-optimal solutions found by the ESA, as shown in the column (PercHypDif ), where error values were between 0.0% and 1.152%.

Experiment 3: Scalability and Evolutionary Hypervolume Improvement
Considering that for the MMAD model, the ESA no longer ended for values of |N | greater than 15, the scalability of the NSGA-II-based algorithm is demonstrated, taking into account that it found solutions for all the experimental instances and it can be easily verified that as the number of elements increases, the NSGA-II-based algorithm gets solutions faster than the ESA (see Tables 4 and 5).
Additionally to the good quality of solutions obtained by the proposed NSGA-II-based algorithm and its scalability for solving large problem instances, it can be observed that the quality of solutions is improved on each generation (or iteration).As shown in Figures 1 and 2, the hypervolume improves (increases) as the number of generations (or iteration) increases.It also can be seen in Figures 3 and Figure 4 how the hypervolume improves towards optimal values.Consequently, the good quality of solutions for large problem instances may also be guaranteed using according number of generations.
In this work, tables with the obtained results considering all objectives and charts of the Pareto front with only two objectives are presented instead of charts or graphs with all objectives, given that the representation

7 Q 8 Qt 11 Pt
Data: |N |, |M |, matrix D = {di,j} Result: Pareto set approximation P known 1 initialize set of solutions P0 2 P 0 = repair unfeasible solutions of P0 3 update set of solutions P known from P 0 4 t = 0; Pt = P 0 5 while is not stopping criterion do 6 Qt = selection of solutions from Pt ∪ P known t = crossover and mutation of solutions of Qt t = repair unfeasible solutions of Q t 9 update set of solutions P known from Q t 10 increment = non-dominated sorting from Pt ∪ Q t 12 end 13 return Pareto set approximation P known • the values of |N | and |M | and • the distances d i,j between each pair of elements represented by a matrix D = {d i,j }.

Figure 4 :
Figure 4: Pareto front vs NSGA-II front in each iteration considering Max-Sum and Max-Min with |N | = 15 for the MMAD model.

Figure 3
Figure 3 shows how the Pareto Front obtained by the NSGA-II-based algorithm is approximated at each iteration to the optimal Pareto Front found by the ESA for the Min-Diff and Min-P-Center Diversity Measures for the MMD model.In the same way for the MMAD model, Figure 4 shows how the Pareto Front obtained by the NSGA-II-based is approximated at each iteration to the Optimal Pareto front for the Max-Sum and Max-Min Diversity Measures.

Table 1 :
[1,2,4]e Matrix d i,j between elements (n i , n j ) in N for Example 1.Considering the above mentioned possible solutions M 1 and M 2 , the best solution taking into account the Max-Sum diversity measure is M 2 , composed by[1,2,4], with 12.28 of diversity.

Table 2 :
Distance Matrix d i,j between elements (n i , n j ) in N for Example 2.

Table 3 :
Correlation coefficients among the results in a problem instance with |N | = 50 and |M | = 5.

Table 4 :
Hypervolume and Elapsed Time for the ESA and NSGA-II for the MMD model.

Table 5 :
Hypervolume and Elapsed Time for the ESA and NSGA-II for the MMAD model.