Part-of-speech tagger for Malay social media texts
Processing the meaning of words in social media texts, such as tweets, is challenging in natural language processing. Malay tweets are no exception because they demonstrate distinct linguistic phenomena, such as the use of dialects from each state in Malaysia; borrowing foreign language terms in...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Penerbit Universiti Kebangsaan Malaysia
2018
|
| Online Access: | http://journalarticle.ukm.my/17663/1/28357-89214-1-PB.pdf http://journalarticle.ukm.my/17663/ https://ejournal.ukm.my/gema/issue/view/1146 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | Processing the meaning of words in social media texts, such as tweets, is challenging in
natural language processing. Malay tweets are no exception because they demonstrate distinct
linguistic phenomena, such as the use of dialects from each state in Malaysia; borrowing
foreign language terms in the context of Malay language; and using mixed languages,
abbreviations and spelling errors or mistakes in sentence structure. Tagging the word class of
tweets is an arduous task because tweets are characterised by their distinctive style, linguistic
sounds and errors. Currently, existing works on Malay part-of-speech (POS) are based only
on standard Malay and formal texts and are thus unsuitable for tagging tweet texts. Thus, a
POS model of tweet tagging for non-standardised Malay language must be developed. This
study aims to design and implement a non-standardised Malay POS model for tweets and
performs assessment on the basis of the word tagging accuracy of test data of unnormalised
and normalised tweet texts. A solution that adopts a probabilistic POS tagging called QTAG
is proposed. Results show that the Malay QTAG achieves best average POS tagging
accuracies of 90% and 88.8% for normalised and unnormalised test datasets, respectively. |
|---|