Customer analysis with machine vision

• Main Author: Tiong, Wei Jie
• Format: Final Year Project / Dissertation / Thesis
• Published: 2023
• Subjects: TJ Mechanical engineering and machinery
• Online Access: http://eprints.utar.edu.my/5824/1/MH_1805100_Final_TIONG_WEI_JIE.pdf; http://eprints.utar.edu.my/5824/

CCTVs usually installed in a business establishment can yield additional customer information, providing valuable insights for marketing analysis. However, manually analyzing the sheer volume of videos can be taxing for humans. Therefore, this study endeavors to develop a computer-vision solution that automates customer analysis on CCTV videos. The proposed solution must be able to fulfil the requirements for customer counting, customer recognition and gender classification. This study aimed to improve the human detection model by eliminating the imperfections in existing models that have a high false rate in detecting the cartoons as humans. These cartoons may be human-like stickers that are placed around retail shops, and false detection may result in inaccurate customer analysis. To evaluate the performance of existing detection models, metrics such as accuracy, precision, recall, F1 score, false detection rate, model size, and parameters are used. To address the issue, the latest algorithms, such as YOLOv5, YOLOv8 and mobilenet ssd, were selected for retraining. The retraining process involved utilization of a dataset consists of 2 classes: human and cartoon, with 11k images per class. The instances in the dataset were well labelled before splitting into train, validation and test sets. Each selected model is then retrained, evaluated and compared to the existing models. The study found that the best model is the retrained YOLOv8n, which achieved a false detection rate of 8.16 %, outperforming all the pretrained models. Meanwhile, it has enhanced the accuracy and F1 score in human detection, improving by 5.38 % and 2.85 % respectively when compared to the best pretrained model, YOLOv8m. Hence, the retrained YOLOv8n has been selected as the human detection model for the proposed solution. When the retrained YOLOv8n detects a customer in the CCTV video, human tracking takes place to track the customer. When the customer passes through a counting line drawn by the system, customer counting occurs, and the system will crop their faces for facial recognition and gender classification. Due to time constraints, several components and algorithms could not be addressed in this study. Future work will focus on improving facial recognition and proposing new methods to explore different approaches.