Implementation of the Omega (ω) index to detect large-scale systematic cheating

Author
Periodical
European Journal of Educational Research
Volume
8
Year
2019
Issue number
4
Page range
1307-1322
Relates to study/studies
PIRLS 2011

Implementation of the Omega (ω) index to detect large-scale systematic cheating

Abstract

Cheating detection is an important issue in standardized testing, especially in large-scale settings. Statistical approaches are often computationally intensive and require specialised software to conduct. We present a two-stage approach that quickly filters suspected groups using statistical testing on an IRT-based answer-copying index. We also present an approach to mitigate data contamination and improve the performance of the index. The computation of the index was implemented through a modified version of an open source R package, thus enabling wider access to the method. Using data from PIRLS 2011 (N=64,232) we conduct a simulation to demonstrate our approach. Type I error was well-controlled and no control group was falsely flagged for cheating, while 16 (combined n=12,569) of the 18 (combined n=14,149) simulated groups were detected. Implications for system-level cheating detection and further improvements of the approach were discussed.