Difficulties in Fair Performance Comparison of Evolutionary Multi-Objective Algorithms

#Benchmarking #evolutionary #algorithms #Multi-objective #optimization #performance #comparison
Share

Professor Hisao Ishibuchi from Southern University of Science and Technology (Sustech) will give a distinguished lecture on fair performance comparison of EMO algorithms.



  Date and Time

  Location

  Hosts

  Registration



  • Add_To_Calendar_icon Add Event to Calendar
  • 100 Fuxing Rd., Gaoxin Campus, University of Science and Technology of China,
  • Hefei, Anhui
  • China 230027
  • Building: Xinzhi Building
  • Room Number: C301

  • Contact Event Host
  • Co-sponsored by University of Science and Technology of China


  Speakers

Hisao Ishibuchi of Southen University of Science and Technology

Topic:

Difficulties in Fair Performance Comparison of Evolutionary Multi-Objective Algorithms

Evolutionary multi-objective optimization (EMO) has been a very active research area in recent years. Almost every year, new EMO algorithms are proposed. When a new EMO algorithm is proposed, computational experiments are conducted in order to compare its performance with existing algorithms. Then, experimental results are summarized and reported as a number of tables together with statistical significance test results. Those results usually show higher performance of the new algorithm than the other examined algorithms. However, fair performance comparison of different EMO algorithms is not easy since the evaluated performance of each algorithm usually strongly depends on experimental settings. In this seminar, we focus on the settings related to the following four issues: (i) termination condition specification, (ii) population size specification, (iii) performance indicator choice, and (iv) test problem choice. First, we clearly demonstrate that each of these issues has strong effects on performance comparison results. That is, totally different comparison results can be obtained from different experimental settings. Then, we discuss how to handle each of these issues for fair performance comparison. These discussions aim to encourage the future development of the EMO research field without focusing too much on the development of overly-specialized new algorithms in a specific setting. Finally, we suggest some promising future research topics related to each issue.

Biography:

Hisao Ishibuchi is a Chair Professor at Southern University of Science and Technology, China. He was the IEEE Computational Intelligence Society (CIS) Vice-President for Technical Activities in 2010-2013 and the Editor-in-Chief of IEEE Computational Intelligence Magazine in 2014-2019. Currently he is an IEEE CIS Administrative Committee Member, an IEEE CIS Distinguished Lecturer, and an Associate Editor of several journals such as IEEE Transactions on Cybernetics and ACM Computing Surveys. He is also General Chair of IEEE WCCI 2024. He received a Fuzzy Systems Pioneer Award from IEEE CIS in 2019, an Outstanding Paper Award from IEEE Transactions on Evolutionary Computation in 2020, an Enrique Ruspini Award for Meritorious Service from IEEE CIS, and Best Paper Awards from FUZZ-IEEE 2009, 2011, EMO 2019, and GECCO 2004, 2017, 2018, 2020, 2021. He also received a JSPS prize in 2007. He is an IEEE Fellow.

Email:

Address:Southern University of Science and Technology, , Shenzhen, China