Validity evidence using expert judgment: a study of using item congruence involving expert judgements for evidence for validity of a reading test

• Main Authors: Jusoh, Zailani, Zubairi, Ainol Madziah, Badrasawi, Kamal J I
• Format: Article
• Language: English; English
• Published: The International Institute of Islamic Thought and Civilization (ISTAC)- IIUM 2018
• Subjects: L Education (General)
• Online Access: http://irep.iium.edu.my/69103/1/69103_Validity%20evidence%20using%20expert%20judgment.pdf; http://irep.iium.edu.my/69103/2/69103_Validity%20evidence%20using%20expert%20judgment_SCOPUS.pdf; http://irep.iium.edu.my/69103/; http://journals.iium.edu.my/shajarah/index.php/shaj/article/view/766

This study demonstrates the use of the Item Objective Congruence Method (IOC) for a content validation procedure to evaluate the congruence between reading test items constructed and the reading sub-skills they were intended to measure. The IOC allows for quantification of the content validity evidence of a test or an instrument. In this study, experts were asked to rate 96 reading comprehension test items against the sub-skills that the items were intended to measure. In line with the concept of justice in Islam, it is important that the evidence of justice is demonstrated. Using ten English language experts who were also experienced test writers, the study found different degrees of agreement between the experts in terms of what some of the items were supposed to measure. Items that showed considerable incongruence are those that test multiple subskills. Whereas, items that had high degree of congruence are those that clearly test a single subskill.