Performance analysis of convolutional neural networks extended with predefined kernels in image classification / Arash Fatehi

• Main Author: Arash , Fatehi
• Format: Thesis
• Published: 2022
• Subjects: QA75 Electronic computers. Computer science; T Technology (General)
• Online Access: http://studentsrepo.um.edu.my/14682/1/Arash_Fatehi.pdf; http://studentsrepo.um.edu.my/14682/2/Arash_Fatehi.pdf; http://studentsrepo.um.edu.my/14682/

While Machine Learning aims to solve more challenging problems, Artificial Neural Networks (ANN) become deeper and more accurate. Convolutional Neural Network (CNN) is not an exception and state-of-art architectures consist of millions of learnable parameters. Aiming for better performance, these networks become more complex and computation intensive. Also, with the rise of IoT devices and edge computing, the importance of model acceleration and reduction of needed computing resources become more curial for training neural networks. Model acceleration and compression techniques often target reducing inference latency and memory usage, and research about reducing the training time was limited to two previous studies. Considering numerous use cases of CNNs, reducing the training time and processing cost is beneficial. CNNs are universal functions and in the case of supervised learning, they will converge to a specific desired function after training. In this research, predefined image processing kernels were merged into CNN's architecture to help the network to converge faster for the use case of image classification. This method can be applied to any classification task of multi-channel sensory data. The efficiency of the method was tested through an experiment on ImageNet, Cifar10, and Cifar100 datasets. The effects on performance were architecture dependent. In the case of CNNs with residual blocks and skip connections, the model was not able to leverage the provided information by image processing filters to converge faster, but CNNs based on VGG had a significantly (up to 125%) faster training time, which is beneficial for training models on embedded devices and edge computing.