A new ontology-based multimodal classification system for social media images of personality traits

• Main Authors: Biswas, Kunal, Shivakumara, Palaiahnakote, Pal, Umapada, Lu, Tong
• Format: Article
• Published: Springer Verlag 2023
• Subjects: QA75 Electronic computers. Computer science; QA76 Computer software; T Technology (General)
• Online Access: http://eprints.um.edu.my/39540/

Number of users of social media is increasing exponentially. People are getting addicted to social media, and because of such addiction, it sometimes causes psychological and mental effects on the users. Understanding user interaction with social media is essential to study personality traits of the users. This paper focuses on classification of personality traits called Big Five factors, namely (i) agreeableness, (ii) conscientiousness, (iii) neuroticism, (iv) extraversion and (v) openness by combining image and textual features through ontology and fully connected neural network (FCNN). The intuition to classify the images of the five classes is that there is a strong correlation between images, profile picture, images of status, text in the images, description of the images uploaded on social media and the person's mind. To extract such observation, we explore an ontology-based approach, which constructs a weighted undirected graph (WUG) based on labels of the images, profile picture, banner image, text in