Skip to Main Content

The quality of each relevant paper was evaluated according to the Quality Assessment Criteria (QAC) defined in the format of questions and presented in Table 2. If a question was addressed by the paper, it could take one of the answers: “yes,” “partly,” and “no,” which received one, one-half, and zero points, respectively. If the question was not addressed at all by the paper, then it received the answer “non-applicable” excluding it from the calculation of the final result. In this way, a paper's final score was calculated by averaging the score of each question. Nine papers had a score [0,0.5] and they were excluded from the study.

Table 2:
Quality assessment questions with possible answers: “Yes,” “Partly,” or “No.”
 QA questions concerning the x-NEAT's principles 
QA1 Are the aims of the research clearly defined? 
QA2 Are the main aspects of the proposed method explained in detail? 
QA3 If the method introduces a new encoding scheme, is it described clearly? If the same encoding as in previous methods is used, is it understandable from the paper? 
 QA questions regarding the experimental procedure 
QA4 Is the experimental procedure clearly described? 
QA5 Is the method evaluated on sufficient number of datasets? (number of datasets 3: yes, 2: partly, 1: no) 
QA6 If the study involves a custom artificial dataset, is its construction method adequately described? If it cannot be described for example, in case of a video game, is the task clearly explained? 
QA7 Are the parameters of the NE algorithm clearly described? 
QA8 Are the metrics used for measuring the algorithm's performance clearly defined? 
QA9 Is each experiment run for an adequate number of repetitions? (Yes: 20, Partly: [10,20), No: [0,10)) 
QA10 Is there a statistical test to test if a statistical difference in the compared performances exists? 
QA11 Is the proposed method compared to the state of the art of NE methods? 
QA12 Is the proposed method compared to other machine learning/EC algorithms? 
 QA questions regarding the reception of the paper from the community 
QA13 Does the study have an adequate number of citations per year?2 (Yes: 1, Partly: [0.5,1), No: [0,0.5)) 
 QA questions concerning the x-NEAT's principles 
QA1 Are the aims of the research clearly defined? 
QA2 Are the main aspects of the proposed method explained in detail? 
QA3 If the method introduces a new encoding scheme, is it described clearly? If the same encoding as in previous methods is used, is it understandable from the paper? 
 QA questions regarding the experimental procedure 
QA4 Is the experimental procedure clearly described? 
QA5 Is the method evaluated on sufficient number of datasets? (number of datasets 3: yes, 2: partly, 1: no) 
QA6 If the study involves a custom artificial dataset, is its construction method adequately described? If it cannot be described for example, in case of a video game, is the task clearly explained? 
QA7 Are the parameters of the NE algorithm clearly described? 
QA8 Are the metrics used for measuring the algorithm's performance clearly defined? 
QA9 Is each experiment run for an adequate number of repetitions? (Yes: 20, Partly: [10,20), No: [0,10)) 
QA10 Is there a statistical test to test if a statistical difference in the compared performances exists? 
QA11 Is the proposed method compared to the state of the art of NE methods? 
QA12 Is the proposed method compared to other machine learning/EC algorithms? 
 QA questions regarding the reception of the paper from the community 
QA13 Does the study have an adequate number of citations per year?2 (Yes: 1, Partly: [0.5,1), No: [0,0.5)) 

Close Modal

or Create an Account

Close Modal
Close Modal