Considering the importance of mathematics in modern society, it is crucial to understand the cognitive processes involved in the acquisition of complex mathematical competency. As a new generation of evaluation theory, cognitive diagnosis has its unique advantages in personalized evaluation. Based on the mathematical cognitive framework of Trends in International Mathematics and Science Study (TIMSS)-2011 and the Chinese mathematics curriculum, this research has formed a mathematical competency model composed of seven cognitive attributes. Sixty-seven released mathematical items for the fourth grade in TIMSS-2011 were used as assessment tools in this study. The deterministic inputs, noisy, “and” gate model was selected as the cognitive diagnosis model in this research, and the parameters of the model were evaluated according to the response data, based on which the effectiveness of the assessment tool was further verified, forming the evaluation framework of students' mathematical competency. This framework was used to analyze the data of 573 students' mathematical competency from Shanghai, China, specifically from three aspects: attribute mastery probability, learning path, and knowledge structure. Results show that students' performance in mastering attributes of mathematical cognition is excellent on the whole; some students' learning paths are leapfrog; there are certain differences in students' knowledge structure despite that they have the same total score. This research is performed as a systematic case study of the evaluation of students' mathematical competency and also provides a new perspective in assessing other knowledge and skills.