ILSA study designs are by nature complex and as a consequence require complex methods of analysis not readily available with most commercial statistical software packages. Of particular interest when conducting analyses of ILSA data is the calculation of sample and imputation variance and the combination of these two variance components. The organizations that conduct ILSAs have accordingly developed specific analytical tools that take these complexities into account.
International Database Analyzer (IEA IDB Analyzer)
The International Database Analyzer (IEA IDB Analyzer), an application developed by the International Association for the Evaluation of Educational Achievement (IEA), can be used to combine and analyze data from IEA studies as well as data from most other large-scale assessments or surveys.
The IEA IDB Analyzer produces SPSS or SAS syntax that can be used to combine and analyze data from across different countries and sources (for example, student, teacher, or school). It requires the user to have direct access to SAS or SPSS and to the databases to be analyzed. The IEA IDB Analyzer is licensed free of cost, not sold, and is for use only in accordance with the terms of the licensing agreement.
The IEA IDB Analyzer consists of two modules – the Merge Module and the Analysis Module – integrated in one single application. It supports the creation of tables and charts and the calculation of averages, percentages, standard deviations, percentiles, linear and logistic regression, and performance levels. Video tutorials on the use of the software are available here.
The Data Visualizer, another tool created by the IEA, can be used for depicting results from TIMSS and PIRLS achievement tests (i.e., scales and sub-scales) for a particular cycle or as a trend over time. The data can be depicted in the form of (interactive) graphs as well as regional or world maps.
International Data Explorer (IDE)
The International Data Explorer (IDE) is a web-based application developed by the Educational Testing Service (ETS) that allows the user to query ILSA databases available online. Unlike with the IEA IDB Analyzer, the user accesses the databases online, and all analyses are performed on a remote computer. There is no direct access to the databases; the user needs only a browser and Internet access.
The IDE allows users to explore performance on achievement tests and results on background questionnaires by running various types of statistical tests. It supports the creation of tables, charts, and maps for displaying averages, percentages, standard deviations, percentiles, and performance levels.
There are several deployments of the IDE covering the various ILSAs. Versions for PIAAC, PIRLS, PISA, and TIMSS are available on the National Center for Education Statistics (NCES) website, and for PIAAC and PISA also on the Organisation for Economic Co-operation and Development (OECD) website. All IDE deployments include data from all countries participating in the studies. The NCES deployments include, in addition to the international databases, questions asked only in the United States.
PISA Data Explorer / Data Analysis Manual
The PISA Data Explorer mentioned above and made available by the OECD allows users to perform customized analyses of PISA data, export data, and create reports. Important technical information on PISA data as well as on their analyses (i.e., the reproduction of the initial results published by the OECD or the performance of secondary analyses) can be found in the PISA Data Analysis Manual and in the PISA Technical Report.