Pre-trained language model with feature reduction and no fine-tuning

• Main Authors: Kit, Y. H., Mokji, M.
• Format: Conference or Workshop Item
• Published: 2022
• Subjects: TK Electrical engineering. Electronics Nuclear engineering
• Online Access: http://eprints.utm.my/id/eprint/98842/; http://dx.doi.org/10.1007/978-981-19-3923-5_59

Pre-trained language models were proven to achieve excellent results in Natural Language Processing tasks such as Sentiment Analysis. However, the number of sentence embeddings from the base model of Bidirectional Encoder from Transformer (BERT) is 768 for a sentence, and there will be more than mil...