Challenges of mining twitter data for analyzing service performance: A case study of transportation service in Malaysia

• Main Authors: Chua, Hui Na, Liao, Alvin Wei Qiang, Low, Yeh Ching, Lee, Angela Siew Hoong, Ismail, Maizatul Akmar
• Format: Conference or Workshop Item
• Published: SPRINGER INTERNATIONAL PUBLISHING AG 2022
• Subjects: QA75 Electronic computers. Computer science
• Online Access: http://eprints.um.edu.my/46290/; https://doi.org/10.1007/978-3-031-04216-4_21

Literature has shown the prominence of extracting social media data to understand public opinion. However, there are little works on how these opportunities can be realized and the challenges in exploiting the opportunities in the transportation industry. Further, data quality and availability using social media may vary according to different demographics due to population size and languages used. Additionally, most of the related prior studies that show the opportunities of using social media data were conducted in North and South America. With this proposition, we seek to investigate the challenges of using Twitter data with text mining techniques for understanding users' opinions and sentiment through a case study of using the data to assess public transportation service performance specifically in a Malaysian context. Our findings indicate that social media data can only be useful in generating reasonable insights if users could input informative words for forming discussed topics to derive opinion, and incline towards a certain sentiment with adjectives. The findings also identified the need for a more proficient dictionary to classify multilingual tweets. Our research provides original evidence proving the potential of using social media data to assess public transportation services performance which may vary depending on the demographics of the social media users.