The effect of rubric on rater’s severity and bias in TVET laboratory practice assessment: analysis using many-facet rasch measurement
Performance assessments such as laboratory practice in Technical and Vocational Education and Training (TVET) are difficult to measure because they are subjective in their characteristics, which can cause bias. The use of a rubric scoring scale is suitable for measuring students’ practical competenc...
Saved in:
Main Authors: | , , , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
Penerbit UTHM
2020
|
Subjects: | |
Online Access: | http://eprints.uthm.edu.my/6376/1/AJ%202020%20%28305%29.pdf http://eprints.uthm.edu.my/6376/ https://doi.org/10.30880/jtet.2020.12.01.006 |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
id |
my.uthm.eprints.6376 |
---|---|
record_format |
eprints |
spelling |
my.uthm.eprints.63762022-01-30T08:04:44Z http://eprints.uthm.edu.my/6376/ The effect of rubric on rater’s severity and bias in TVET laboratory practice assessment: analysis using many-facet rasch measurement Ab Rahman, Azmanirah Muhamad Hanafi, Nurfirdawati Yusof, Yusmarwati Mukthar, Marina Ibrahim Awang, Halizah Mohamed Yusof, Anizam T Technology (General) LC1041-1048 Vocational education (General) Performance assessments such as laboratory practice in Technical and Vocational Education and Training (TVET) are difficult to measure because they are subjective in their characteristics, which can cause bias. The use of a rubric scoring scale is suitable for measuring students’ practical competency as it could translate the qualitative criteria into quantitative forms on a grading scale. The purpose of this study is to analyse the effect of rater severity for the performance of component installation of an electronic circuit project using Many-Facet Rasch Measurement (MFRM). Nine raters have examined the competency performance of 68 students using a four-point rating scale rubric consisting of 16 items. This study was conducted at three vocational colleges. The results showed that all raters measured with different severities but very high consistency. The effect of rater severity did not exist at the group level but exhibited at the individual level. It can also be examined that all raters were biased on the item and also towards students, either by being severe or lenient. Thus, the rubric cannot eliminate the effect of bias during the assessment. However, it can help raters to be consistent so that they have the same understanding when translating the rubric. These findings have implications for the psychometric quality of the electronic circuit project performance assessment. Penerbit UTHM 2020 Article PeerReviewed text en http://eprints.uthm.edu.my/6376/1/AJ%202020%20%28305%29.pdf Ab Rahman, Azmanirah and Muhamad Hanafi, Nurfirdawati and Yusof, Yusmarwati and Mukthar, Marina Ibrahim and Awang, Halizah and Mohamed Yusof, Anizam (2020) The effect of rubric on rater’s severity and bias in TVET laboratory practice assessment: analysis using many-facet rasch measurement. Journal of Technical Education and Training, 12 (1). pp. 57-67. ISSN 2229-8932 https://doi.org/10.30880/jtet.2020.12.01.006 |
institution |
Universiti Tun Hussein Onn Malaysia |
building |
UTHM Library |
collection |
Institutional Repository |
continent |
Asia |
country |
Malaysia |
content_provider |
Universiti Tun Hussein Onn Malaysia |
content_source |
UTHM Institutional Repository |
url_provider |
http://eprints.uthm.edu.my/ |
language |
English |
topic |
T Technology (General) LC1041-1048 Vocational education (General) |
spellingShingle |
T Technology (General) LC1041-1048 Vocational education (General) Ab Rahman, Azmanirah Muhamad Hanafi, Nurfirdawati Yusof, Yusmarwati Mukthar, Marina Ibrahim Awang, Halizah Mohamed Yusof, Anizam The effect of rubric on rater’s severity and bias in TVET laboratory practice assessment: analysis using many-facet rasch measurement |
description |
Performance assessments such as laboratory practice in Technical and Vocational Education and Training (TVET) are difficult to measure because they are subjective in their characteristics, which can cause bias. The use of a rubric scoring scale is suitable for measuring students’ practical competency as it could translate the qualitative criteria into quantitative forms on a grading scale. The purpose of this study is to analyse the effect of rater severity for the performance of component installation of an electronic circuit project using Many-Facet Rasch Measurement (MFRM). Nine raters have examined the competency performance of 68 students using a four-point rating scale rubric consisting of 16 items. This study was conducted at three vocational colleges. The results showed that all raters measured with different severities but very high consistency. The effect of rater severity did not exist at the group level but exhibited at the individual level. It can also be examined that all raters were biased on the item and also towards students, either by being severe or lenient. Thus, the rubric cannot eliminate the effect of bias during the assessment. However, it can help raters to be consistent so that they have the same understanding when translating the rubric. These findings have implications for the psychometric quality of the electronic circuit project performance assessment. |
format |
Article |
author |
Ab Rahman, Azmanirah Muhamad Hanafi, Nurfirdawati Yusof, Yusmarwati Mukthar, Marina Ibrahim Awang, Halizah Mohamed Yusof, Anizam |
author_facet |
Ab Rahman, Azmanirah Muhamad Hanafi, Nurfirdawati Yusof, Yusmarwati Mukthar, Marina Ibrahim Awang, Halizah Mohamed Yusof, Anizam |
author_sort |
Ab Rahman, Azmanirah |
title |
The effect of rubric on rater’s severity and bias in TVET laboratory practice assessment: analysis using many-facet rasch measurement |
title_short |
The effect of rubric on rater’s severity and bias in TVET laboratory practice assessment: analysis using many-facet rasch measurement |
title_full |
The effect of rubric on rater’s severity and bias in TVET laboratory practice assessment: analysis using many-facet rasch measurement |
title_fullStr |
The effect of rubric on rater’s severity and bias in TVET laboratory practice assessment: analysis using many-facet rasch measurement |
title_full_unstemmed |
The effect of rubric on rater’s severity and bias in TVET laboratory practice assessment: analysis using many-facet rasch measurement |
title_sort |
effect of rubric on rater’s severity and bias in tvet laboratory practice assessment: analysis using many-facet rasch measurement |
publisher |
Penerbit UTHM |
publishDate |
2020 |
url |
http://eprints.uthm.edu.my/6376/1/AJ%202020%20%28305%29.pdf http://eprints.uthm.edu.my/6376/ https://doi.org/10.30880/jtet.2020.12.01.006 |
_version_ |
1738581485141622784 |
score |
13.188404 |