Comparison of Test Scores Reliability Coefficients under a Set of Variables: A Monte Carlo Simulation Study

Document Type : Original Article

Author

Assistant professor of educational psychology Department of Psychology - College of Education Qassim University - Kingdom of Saudi Arabia

Abstract

In this study, a comparison was made between the values ​​and relative bias ratios of thirteen different types of reliability coefficients, which are the Guttman reliability lower bounds coefficients (λ1, λ2, λ3, λ4, λ5, λ6), and the Cronbach alpha reliability coefficient (α), and the Total Omega coefficient (ωt), Hierarchical Omega-asymptotic coefficient (ωh), Greatest lower bound coefficient (glb), Polychoric alpha coefficient (αpoly), Worst split half coefficient (β), Alpha-stratified coefficient (αstrata), and Maximal Reliability Coefficient (H) through data generated by Monte Carlo method with two measurement frames unidimensional and multidimensional models through four data conditions (Data Type - Response Options - Test Length - Sample Size), and it became clear from the results that there are four types of reliability coefficients that give the highest estimation coefficient of measurement reliability and can be considered non-bias reliability coefficients as it outperforms the traditional Cronbach alpha coefficient, making it the best for use in different data cases, namely the total omega reliability coefficient (ωt), the greatest lower bound reliability coefficient (glb), the maximum Guttman lower bound reliability coefficient (λ4), and the polychoric alpha reliability coefficient (αpoly). The best performance among them was Total Omega Coefficient performance (ωt), which gave the highest values ​​of reliability with the lowest relative bias for the majority of interactions between cases of data types and tests, and the diversity of sample sizes.

Keywords

Main Subjects