There has been a strong narrative in Australia of falling attainment in high school science, with much of the campaign informed by results from international standardised tests such as Programme for International Student Assessment (PISA), which shows a year-on-year decline in scientific literacy of Australian 15-year-old students. These results have been used to justify significant policy and curriculum reform, despite the known limitations of PISA and a lack of additional evidence to support this decline in other tests. In this paper, results from standardised tests administered in Australia will be compared to create a fulsome picture of attainment for high school science students. Reports include both the compilation of data from existing reports and new analyses. With the latest (2018/9) reports from PISA, Trends in International Mathematics and Science Study (TIMSS), and National Assessment Program for Scientific Literacy (NAP-SL) (an Australian test of Science Literacy) and data shared by the NSW Department of Education on ‘The Validation of Assessment for Learning and Individual Development’ (VALID) test for the years 2015, 2016, 2017, and 2018, this offers the most complete picture of student attainment in science to date. Results show that there are disagreements between tests on cohort achievement over time and distribution of attainment at different ‘proficiency levels’. These results suggest caution when using these key results from these tests to inform policy and pedagogy.