Image-based virtual try-on system using deep learning

• Main Author: Gan, Yong Hao
• Format: Final Year Project / Dissertation / Thesis
• Published: 2021
• Subjects: QA75 Electronic computers. Computer science; T Technology (General)
• Online Access: http://eprints.utar.edu.my/4288/1/18ACB00330_FYP.pdf; http://eprints.utar.edu.my/4288/

Due to the pandemic, there are more and more people shopping online especially for buying cloths. However, online shopping does not allow physical try-on, therefore limiting customer understanding of how a cloth will look on them. Thus, the image based virtual try-on system is introduced to allow customer to able to try on the desired cloth from online store. The aim of this project is to research and make an improvement to the existing system like CP-VTON. The system able to generate high fidelity try-on images that preserves the overall appearance and the characteristics of clothing items. Not only that, the system also able to preserve other human components other than the target clothing area by adding face, hair, lower cloths, and legs. The system consists of two modules, the Geometric Matching Module (GMM) and Try-On Module (TOM). To warp in-shop clothing item to the desired image of a person with high accuracy in GMM, grid interval consistency loss and an occlusion handling technique are proposed. Grid interval consistency loss regularizes transformation to prevent distortion of patterns in clothes and an occlusion handling technique encourages proper warping despite target bodies are covered by hair or arms. After that for TOM, face, hair, lower cloths, and legs are added to person representation input of TOM to preserve other human components other than the target clothing area. TOM will synthesize the final try-on image of the target person seamlessly with the warped clothes from GMM and the person representation. Lastly a synthesizing discriminator is also added to the end of TOM using SN-PatchGAN to further improve the quality of images generated.