Intelligent Malaysian Sign Language Translation System Using Convolutional-Based Attention Module with Residual Network

• Main Authors: Rehman Ullah, Khan, Hizbullah, Khattak, Sheng Wong, Woei, Hussain, AlSalman, Mogeeb A. A., Mosleh, Sk. Md. Mizanur, Rahman5
• Format: Article
• Language: English
• Published: Hindawi 2021
• Subjects: Universiti Malaysia Sarawak > Periodicals > Bulletin; P Philology. Linguistics
• Online Access: http://ir.unimas.my/id/eprint/37106/1/residual1.pdf; http://ir.unimas.my/id/eprint/37106/; https://www.hindawi.com/journals/cin/2021/9023010/; https://doi.org/10.1155/2021/9023010

(e deaf-mutes population always feels helpless when they are not understood by others and vice versa. (is is a big humanitarian problem and needs localised solution. To solve this problem, this study implements a convolutional neural network (CNN), convolutional-based attention module (CBAM) to recognise Malaysian Sign Language (MSL) from images. Two different experiments were conducted for MSL signs, using CBAM-2DResNet (2-Dimensional Residual Network) implementing “Within Blocks” and “Before Classifier” methods. Various metrics such as the accuracy, loss, precision, recall, F1-score, confusion matrix, and training time are recorded to evaluate the models’ efficiency. (e experimental results showed that CBAM-ResNet models achieved a good performance in MSL signs recognition tasks, with accuracy rates of over 90% through a little of variations. (e CBAM-ResNet “Before Classifier” models are more efficient than “Within Blocks” CBAM-ResNet models. (us, the best trained model of CBAM-2DResNet is chosen to develop a real-time sign recognition system for translating from sign language to text and from text to sign language in an easy way of communication between deaf-mutes and other people. All experiment results indicated that the “Before Classifier” of CBAMResNet models is more efficient in recognising MSL and it is worth for future research.