Immersive soft skills training application using large language models and virtual reality

• Main Author: Ng, Jing Ying
• Format: Final Year Project / Dissertation / Thesis
• Published: 2024
• Subjects: T Technology (General); TD Environmental technology. Sanitary engineering
• Online Access: http://eprints.utar.edu.my/6667/1/fyp_CS_2024_NJY.pdf; http://eprints.utar.edu.my/6667/

As Malaysia undergoes a significant transition towards a skills-based economy, there is an increasing demand for soft skills training courses as individuals seek to gain their job competencies. An immersive soft skills training application involves delivering scenario-based simulation practices and personalized feedback. However, existing Virtual Reality (VR) training applications are still struggling to balance cost-effectiveness, cognitive realism and comprehensive evaluation. This is because most existing applications rely on a decision-tree approach, where the storyline is constrained by preset branching choices. This not only requires a lot of human input to complete the storyline, but the overall experience still lacks cognitive realism. Besides, most existing applications only rely on quantitative metrics for evaluation, which fall short of providing comprehensive feedback in terms of soft skills training. In this project, the main objective is to develop an immersive soft skills training application using Large Language Models (LLMs) and VR. In short, we have demonstrated the capability of LLMs in generating human-like behaviours. Besides, the combination of quantitative and qualitative data has improved the comprehensiveness of the evaluation process.