Language has been widely acknowledged as a determining factor in mathematical achievement. Less understood, however, is the relationship between students' language and their performance on tests of mathematics when taking into consideration the presence of mathematical difficulties. We investigated the effects of two different language systems, Chinese and English, on the mathematical performance of fourth-grade (or age equivalent) students (N = 23,220) with varying levels of demonstrated mathematical and reading ability. For this investigation, we used a subset of the 2011 Progress in International Reading and Literacy Study (PIRLS) and Trends in International Mathematics and Science Study (TIMSS) from students who were tested in Chinese or English in nine countries. Findings from hierarchical linear modeling (HLM) analyses revealed that the main effect of language on mathematical performance remained significant once variables for mathematical ability were added to the model. Further, significant language-by-mathematical ability interactions were observed when controlling for country, gender, maternal education, and age. Thus, the effect of language on mathematical performance may be especially salient in the presence of mathematical difficulties. Implications of these findings include the need for further investigations of language and its effects on mathematical performance for Chinese- and English-speaking students in order to clarify how this relationship may vary within specific language populations.