Accuracy Performance and Potentiality of Real-Time Avatar Lip Sync Animation in Different Languages

• Main Authors: Loh, Ngiik Hoon, Siti Shukhaila, Shaharuddin
• Format: Article
• Language: English
• Published: Blue Eyes Intelligence Engineering & Sciences Publication 2019
• Subjects: L Education (General); T Technology (General)
• Online Access: http://ir.unimas.my/id/eprint/24719/1/Loh%2C%20Ngiik%20Hoon.pdf; http://ir.unimas.my/id/eprint/24719/; http://www.ijrte.org

With the fast growth in computing power nowadays, the qualities of animation enable an extra layer of visually con-vincing realism. In lip sync animation, creation of realistic lip movement is arduous in getting the lip shape and position to syn-chronize with the speech sounds.To note, spendinghours in manually generating every single lip movement can be a long and challengingtask.Consequently,acomprehensive analysis on viseme based multiple phonemes in English, Bahasa Melayu and Mandarin was carried out, to develop anaccurate and potential platform forrealtime talkingavatar inmultiple languages.The accuracy performancebetween human and avatar in real timewere compared and evaluated. The findings revealed successfulutilization of real time synchronization to drive the synthetic 3D avatars based on live speechinputfor multiple languages, with satisfiedaccurate lip motion result.This paper provides useful knowledge for multilingual solution which accurately predicts mouth movement on real human face, when a person is speaking and directs to lip sync process. Itcontributesto live performances and valuable inopen-ended fieldwith tons of potential, such as animation productionindustry, entertainment, gaming, digital marketingand media education.