Enhancing rare disease diagnosis: a weighted cosine similarity approach for improved k-nearest neighbor algorithm

• Main Authors: Abokadr, Somiya, Azman, Azreen, Hamdan, Hazlina, Amelina, Nurul
• Format: Article
• Published: Little Lion Scientific 2023
• Online Access: http://psasir.upm.edu.my/id/eprint/107707/; http://www.jatit.org/

Diagnosing rare diseases is challenging because they affect only a restricted group of individuals, usually identified as one out of every 2,000 people within the European Union and no more than one out of 1,250 individuals in the United States. This makes it difficult for doctors to recognize the symptoms of these diseases. This paper focuses on the challenges of diagnosing rare diseases due to their low prevalence rates and difficulties in recognizing their symptoms. Machine learning techniques often face difficulties in classifying patients with rare diseases because of their small sample sizes, leading to biased results. They proposed a weighted cosine similarity approach as a distance measure for the k-nearest neighbours algorithm instead of the conventional cosine similarity to address this issue. The use of genetic optimization to select the best weights for the weighted cosine similarity. The Rare Metabolic Diseases Database was used as a case study, and the results demonstrated that reducing the classification bias between majority and minority classes improves all classification performance measures. However, as the number of classes and imbalance ratio increase, the approach's effectiveness decreases, eventually reaching zero. Future work will focus on reformulating the g-mean to smooth its values and avoid assigning a zero score when all class instances are misclassified.