Comparative Study of Surrogate Techniques for CNN Hyperparameter Optimization

Optimizing hyper parameters in Convolutional Neural networks is a tedious process for many researchers and practitioners. It requires a high degree of expertise or experience to optimise the hyper parameters, and manual optimisation is likely to be biased. To date, methods or approaches to automate...

Full description

Saved in:
Bibliographic Details
Main Authors: Mohd Aszemi, Nurshazlyn, M. Zakaria, Nordin, Paneer Selvam, Dhanapal Durai Dominic
Format: Book Section
Language:English
Published: Computing & Intelligent Systems, SCRS 2022
Subjects:
Online Access:http://utpedia.utp.edu.my/id/eprint/24082/1/Comparative%20Study%20of%20Surrogate%20Techniques%20for%20CNN%20Hyperparameter%20Optimization.pdf
https://doi.org/10.52458/978-81-95502-00-4-48
http://utpedia.utp.edu.my/id/eprint/24082/
https://www.publications.scrs.in/chapter/978-81-95502-00-4/48
https://doi.org/10.52458/978-81-95502-00-4-48
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Optimizing hyper parameters in Convolutional Neural networks is a tedious process for many researchers and practitioners. It requires a high degree of expertise or experience to optimise the hyper parameters, and manual optimisation is likely to be biased. To date, methods or approaches to automate hyper parameter optimization include grid search, random search, and Genetic Algorithms (GAs). However, evaluating large number of sample points in the hyperparameter configuration space, as is typically required by these methods, is computationally expensive process. Hence, the objective of this paper is to explore regression as a surrogate technique in CNN hyperparameter optimisation. Performance in terms of accuracy, error rate, training time and coefficient of determination (R2) are evaluated and recorded. Although there is no significant performance difference between the resulting optimized Deep Learning and state-of-the-art on CIFAR-10 datasets, using regression as a surrogate technique for CNN hyperparameter optimization contributes to minimising the time taken for the optimization process, a benefit which has not been fully explored in the literature to the best of the author’s knowledge.