The relationship between response-time effort and accuracy in PISA science multiple choice items

Periodical
International Journal of Testing
Volume
20
Year
2020
Issue number
3
Page range
187-205
Relates to study/studies
PISA 2015

The relationship between response-time effort and accuracy in PISA science multiple choice items

Abstract

The study examined the relationship between examinees’ test-taking effort and their accuracy rate on items from the PISA 2015 assessment. The 10% normative threshold method was applied on Science multiple-choice items in the Cyprus sample to detect rapid guessing behavior. Results showed that the extent of rapid guessing across simple and complex multiple-choice items was on average less than 6% per item. Rapid guessers were identified, and for most items their accuracy was lower than the accuracy for students engaging in solution-based behavior. A number of plausible explanations were graphically evaluated for items for which accuracy was higher for the rapid guessing subgroup. Overall, this empirical investigation presents original evidence on test-taking effort as measured by response time in PISA items and tests propositions of Wise’s (2017) Test-Taking Theory.