Mask segmentation and classification with enhanced grasshopper optimization of 3D hand gestures

• Main Author: Salam Khan, Fawad
• Format: Thesis
• Language: English; English; English
• Published: 2023
• Subjects: T Technology (General)
• Online Access: http://eprints.uthm.edu.my/10831/1/24p%20FAWAD%20SALAM%20KHAN.pdf; http://eprints.uthm.edu.my/10831/2/FAWAD%20SALAM%20KHAN%20COPYRIGHT%20DECLARATION.pdf; http://eprints.uthm.edu.my/10831/3/FAWAD%20SALAM%20KHAN%20WATERMARK.pdf; http://eprints.uthm.edu.my/10831/

The difficulties associated with extracting 3D hand meshes from depth image utilizing 2D convolutional neural networks. The precision of such estimations is frequently hampered by visual distortions caused by nonrigidity, complex backdrops, and shadows. This research provides a unique methodology that combines the enhanced grasshopper optimization method for feature optimization with MASK-RCNN and FCN for segmenting and classifying 3D hand gestures to address these problems. In order to evaluate the proposed method, a 3D gesture data set is generated. In addition, a skeleton model for RGB hand gestures is constructed by estimating the degree of freedom (DoF) using human kinematics. The segmentation of 3D hand gestures is computed using the ResNet50 backbone network, and the Overlap Coefficient (OC) is employed as an evaluation metric. On the other hand, the ResNet101 backbone network is used to calculate the classification of 3D hand gestures. Experimental results reveal that the proposed method achieves greater accuracy in segmenting and classifying 3D hand gestures than existing methods. The study also emphasizes the significance of using feature optimization approaches and developing skeletal models to estimate (DoF) in order to improve the precision of 3D hand gesture analysis. This study provides a promising approach for robust and precise 3D hand gesture recognition, with potential applications in disciplines such as human-computer interaction and virtual reality. The test results show best accuracy for 3D hand gesture classification and segmentation. On the private dataset, the classification accuracy is 99.05 %, whereas 99.29 % on the Kinect dataset, 99.39 % and 99.29% using SKIG and ChaLearn dataset during validation. The OC is 88.16 % and 88.19 %, respectively which is the highest available accuracy compared with other methods. The mAP of ChaLearn 93.26%, private 81.48%, SKIG 75.21% and Kinect 66.74%.