MULTIMODAL PERSON RECOGNITION SYSTEM USING CONVOLUTIONAL NEURAL NETWORK

• Main Author: JAGATHIS, KARUNAKARAN
• Format: Final Year Project Report
• Language: English; English
• Published: Universiti Malaysia Sarawak, (UNIMAS) 2022
• Subjects: TK Electrical engineering. Electronics Nuclear engineering
• Online Access: http://ir.unimas.my/id/eprint/40112/1/Jagathis%20AL%20Karunakaran%2024pgs.pdf; http://ir.unimas.my/id/eprint/40112/4/Jagathis%20AL%20Karunakaran%20ft.pdf; http://ir.unimas.my/id/eprint/40112/

Multimodal person recognition system in the current age is an alternative to unimodal biometric system. Recognition systems routinely used for person identification in various segments and commonly resort to daily life use. In this project, two pretrained Convolutional Neural Networks were trained, tested and evaluated using MATLAB R2021b. The networks are AlexNet with 25 layers and VGG16 with 16 layers. Then an optimal network was opted, AlexNet and used to train face and fingerprint dataset separately with different variation of hyperparameter. The face and fingerprint dataset used to train and test the networks are self-created face dataset and NIST Special Database 302 fingerprint dataset. The accuracy of the AlexNet for face and fingerprint were 95.00% and 98.67% respectively. The AlexNet model was evaluated with an accuracy of 98.67% and 100% in the 5-fold Validation test. The accuracy of the confusion matrix was the same for the AlexNet networks of face and fingerprint. The face and fingerprint networks were later fused in decision-level fusion to produce an overall multimodal recognition network. AlexNet has an average high accuracy of 96.84% setting a high standard for future work in multimodal person recognition suggesting AlexNet as an effective pretrained network for classification.