Benchmarks for evaluating optimization algorithms and benchmarking MATLAB derivative-free optimizers for practitioners’ rapid access

Li, L., Saldivar, A. A. F., Bai, Y., Chen, Y., Liu, Q. and Li, Y. (2019) Benchmarks for evaluating optimization algorithms and benchmarking MATLAB derivative-free optimizers for practitioners’ rapid access. IEEE Access, 7, pp. 79657-79670. (doi: 10.1109/ACCESS.2019.2923092)

Abstract

MATLAB ® has built in five derivative-free optimizers (DFOs), including two direct search algorithms (simplex search, pattern search) and three heuristic algorithms (simulated annealing, particle swarm optimization, and genetic algorithm), plus a few in the official user repository, such as Powell's conjugate (PC) direct search recommended by MathWorks ® . To help a practicing engineer or scientist to choose a MATLAB DFO most suitable for their application at hand, this paper presents a set of five benchmarking criteria for optimization algorithms and then uses four widely adopted benchmark problems to evaluate the DFOs systematically. Comprehensive tests recommend that the PC be most suitable for a unimodal or relatively simple problem, whilst the genetic algorithm (with elitism in MATLAB, GAe) for a relatively complex, multimodal or unknown problem. This paper also provides an amalgamated scoring system and a decision tree for specific objectives, in addition to recommending the GAe for optimizing structures and categories as well as for offline global search together with PC for local parameter tuning or online adaptation. To verify these recommendations, all the six DFOs are further tested in a case study optimizing a popular nonlinear filter. The results corroborate the benchmarking results. It is expected that the benchmarking system would help select optimizers for practical applications.

• Item Type: Articles
• Additional Information: This work was supported in part by the Guangdong Higher Education Research Program under Grant 2017KQNCX193, in part by the Dongguan University of Technology under Research Grant ‘Industry 4.0 Smart Design and Innovation Platform’ KCYKYQD2017014 and under Postdoctoral Research Start-Up Grant GC300501-18, and in part by the National Natural Science Foundation of China under Grant 71801044 and Grant 61773119.
• Status: Published
• Refereed: Yes
• Glasgow Author(s) Enlighten ID: Li, Professor Yun
• Authors: Li, L., Saldivar, A. A. F., Bai, Y., Chen, Y., Liu, Q., and Li, Y.
• College/School: College of Science and Engineering > School of Engineering > Systems Power and Energy
• Journal Name: IEEE Access
• Publisher: IEEE
• ISSN: 2169-3536
• ISSN (Online): 2169-3536
• Published Online: 14 June 2019
• Copyright Holders: Copyright © 2019 IEEE
• First Published: First published in IEEE Access 7: 79657-79670
• Publisher Policy: Reproduced in accordance with the publisher copyright policy