Deep reinforcement learning approaches for multi-objective problem in Recommender Systems

• Main Author: Ee, Yeo Keat
• Format: Thesis
• Language: English
• Published: 2022
• Subjects: Reinforcement learning; Recommender systems (Information filtering)
• Online Access: http://psasir.upm.edu.my/id/eprint/113135/1/113135.pdf; http://psasir.upm.edu.my/id/eprint/113135/

Most of the recommender system merely focus on accuracy of rating prediction or recommendation of trendy items. Nonetheless, other non-accuracy metrics such as novelty and diversity should not be neglected to provide quality recommendation. The current major existing multi-objective recommendation approaches utilize collaborative filtering method as rating predictor to replenish the missing ratings and combined with evolutionary algorithm for only bi-objective optimization. However, collaborative filtering suffers from cold-start problem and incapable to predict rating on highly sparse user-item matrix besides difficulty to incorporate side features information such as user latent, which led to weak performance when encountering new items or users. On the other hand, the evolutionary algorithm is notorious with premature convergence issue and suffering from curse of dimensionality. This study proposes deep reinforcement learning approaches based on Deep Q-Network to improve multi-objective optimization in recommendation environment and investigated its capability to optimizing precision, novelty, and diversity concurrently. The MovieLens 100k dataset is applied to evaluate the performance of the proposed approaches, which do not require separate rating predictor such as done in benchmarked works. This is because the reinforcement learning agent is able to predict items directly by capture user latent information and explore large sparsity state space effectively. The experiment results demonstrated that embedding user latent features contributed to quality improvement in terms of precision by 19.80%, and novelty as well as diversity, by 20.46% and 1.60% respectively. Besides that, the experiment shows that agent which learning sequential data has earned lower precision by 17.57% and novelty by 4.68% compared to the agent that without learning sequential data, however, it achieved better diversity by 2.66%. In the performance comparison between proposed deep reinforcement learning with evolutionary algorithm, despite one of the variants of evolutionary algorithm has good performance in precision, it has rather weak performance in term of novelty and diversity. In contrast, the proposed approaches obtained better novelty and diversity results compared to evolutionary algorithm with sacrificing a certain degree of precision. Overall, the deep reinforcement learning approaches are able to recommend accurate item concurrently with achieving good diversity and novelty as well.