Hierarchical approach for fusion of electroencephalography and electromyography for predicting finger movements and kinematics using deep learning

• Main Authors: Tanaya Das, Tanaya Das, Gohain, Lakhyajit, Kakoty, Nayan M., Malarvili, M. B., Prihartini Widiyanti, Prihartini Widiyanti, Kumar, Gajendra
• Format: Article
• Published: Elsevier B.V. 2023
• Subjects: Q Science (General); T Technology (General); TA Engineering (General). Civil engineering (General)
• Online Access: http://eprints.utm.my/106033/; http://dx.doi.org/10.1016/j.neucom.2023.01.061

The brain is a unique organ that performs multiple processes simultaneously, such as sensory, motor, and cognitive function. However, several neurological diseases (ataxia, dystonia, Huntington's disease) or trauma affect the limb movement and there is no cure. Although brain-computer interfaces (BCIs) have been recently used to improve the quality of life for people with severe motor disabilities, anthropomorphic control of a prosthetic hand in upper limb rehabilitation still remains an unachieved goal. To this purpose, a hierarchical integration of neural commands to fingers was applied for execution of human hand grasping with better precision. For finger movement prediction and kinematics estimation, a neuromuscular approach was employed to establish a hierarchical synergy between electroencephalography (EEG) and electromyography (EMG). EEG, EMG and metacarpophalangeal (MCP) joint kinematics were acquired during five finger flexion movements of the human hand. EMG for five finger movements and kinematics were estimated from EEG using linear regression. A Long Short-Term Memory network (LSTM) and a random forest regressor were adjoined hierarchically for prediction of finger movements and estimation of finger kinematics from the estimated EMG. The results showed an average accuracy of 84.25 ± 0.61 % in predicting finger movements and an average minimum error of 0.318 ± 0.011 in terms of root mean squared error (RMSE) in predicting finger kinematics from EEG across six subjects and five fingers. These findings suggest the implementation of a hierarchical approach to develop anthropomorphic control for upper limb prostheses.