Selective nonresponse can introduce bias in longitudinal surveys. The present study examines the role of cognitive skills (more specifically, literacy skills), as measured in large-scale assessment surveys, in selective nonresponse in longitudinal surveys. We assume that low-skilled respondents perceive the cognitive assessment as a higher burden than higher-skilled respondents because they are more likely to experience negative feelings. We hypothesize that low-skilled respondents are more likely than high-skilled respondents to refuse to participate in a follow-up wave. We analysed data from two assessment surveys in Germany with similar study designs, target populations, and assessment instruments. Results provide support for our hypothesis. Consistently across both surveys, respondents with the lowest literacy skills had a higher probability of refusal than those with the highest literacy skills. This difference persisted even after controlling for several established predictors of nonresponse, including education.