From Item Analysis to Assessment Analysis: Introducing New Formulae

• Main Authors: Puthiaparampil, Thomas, Rahman, M.R., Fong, Isabel Lim
• Format: E-Article
• Language: English
• Published: AMEE MedEdPublish 2017
• Subjects: L Education (General)
• Online Access: http://ir.unimas.my/id/eprint/16234/1/From%20Item%20Analysis%20to%20Assessment%20Analysis%20%28abstract%29.pdf; http://ir.unimas.my/id/eprint/16234/; https://www.mededpublish.org/manuscripts/760/v1

Item analysis of individual multiple choice questions have been widely used for several decades. But formulae for analysis of manually-marked assessments are lacking. Evaluation and comparison of such assessments used in medical schools remain a guess work. In this study we have introduced new formulae aligned with item analysis formulae, which can be used to analyse all assessment methods. While the existing formulae use a binary (pass/fail) criterion, the new formulae have used actual scores. While the existing formulae use the scores of a single assessment to rank the candidates to determine the high and low scorers for calculating the discrimination index (DISi), the new formula used the grand total scores of the entire examination for it. It is claimed that the new formulae, utilizing actual scores, would make the indexes more realistic. Eight examinations, each comprising nine assessments, were used to validate this claim. The DISi and Difficulty Index (DIFi) were calculated using the binary formulae and the new formulae. Comparisons were made using parametric and non-parametric tests. The ensued positive correlation of indexes indicated that the new formulae are feasible, realistic and easy to apply. More replicates are required to prove their validity and reliability.