SEA-PLM 2019 Design
Overall approach to data collection
- Proctored assessment of student achievement
- Self-administered questionnaires for students, parents, teachers, and school principals
All children enrolled in the grade that represents 5 years of schooling counting from the first year of ISCED Level 1
Stratified two-stage cluster sampling design
First Stage: Sampling schools
- Schools were selected systematically with selection probability proportional to the number of enrolled Grade 5 children from the targeted population.
- A minimum of 150 schools were sampled from each participating country.
Second Stage: Sampling classes within schools
- One Grade 5 class was selected randomly within each sampled school.
- All children of the selected class were sampled.
Per country (achieved)
- Between 149 and 233 schools
- One Grade 5 class per school
- Between 4,800 and more than 6,000 students
Across all countries: more than 31,000 students
Student assessment in mathematics, reading, and writing
- Paper-and-pencil tests
- Types of questions
- Selected-response questions (four-option multiple-choice questions; in the mathematics and reading tests)
- Constructed-response questions, with short responses (a number or solution in the mathematics test and a word or up to one or two sentences in the reading test)
- Writing questions (in the writing test only; with responses ranging in length from a single word to multiple paragraphs)
- Rotated booklet design
- Eighteen different test booklets administered randomly in each sampled classroom and school
- Each child had 1 hour to complete the test (2 out of the 3 domains in each booklet).
- For students, it was administered as a separate module to the student questionnaire and without any time constraints.
- For teachers, the respective questions were integrated in the background questionnaire.
- For both
- Likert-style item response format
Four questionnaires, all in paper-and-pencil format:
- Student questionnaire
- To be completed by students after the test
- 30 minutes for completion of the questionnaire
- Parent questionnaire: to be completed by the parents of participating children
- School questionnaire: to be completed by school principals
- Teacher questionnaire: to be completed by Grade 5 teachers
- achievement test
Assessment instruments administered in 8 different languages:
- Khmer: Cambodia
- Lao language: Lao PDR
- Chinese, Malay, Tamil: Malaysia
- Myanmar language: Myanmar
- English: Philippines
- Vietnamese: Viet Nam
- Source versions of all SEA-PLM 2019 materials were prepared in English.
- English-language source materials were separately translated by two different translators into the target language.
- The two translations were reconciled into a single national version by a third person.
- The quality of the instruments and translations were pre-tested during a field trial in each participating country.
- The process was monitored and approved by each country and verified by an external body in collaboration with the National Team based on standardized procedures.
Measures during data collection
- In each country, the National Team was responsible for managing the data-collection operations and all activities conducted at the national level, such as printing, coding, and data entry.
- The National Teams prepared and implemented the survey and supervised all operations and contributors using SEA-PLM guidelines and national resources.
- The quality of the instruments was pre-tested during a field trial in each participating country on children from a sample of schools representing different national contexts.
- People involved in the survey signed a confidentiality agreement to restrict access to SEA-PLM materials.
- All administrators were trained by the National Team prior to operations, following SEA-PLM 2019 standardized materials, manuals, and procedures.
- ACER technical experts supported National Teams in all operations by delivering in-country and remote assistance.
- Countries reported these operations to the SEA-PLM Secretariat for memory and quality assurance.
Measures during data processing and cleaning
- Technical standards were defined and agreed upon by participating countries and experts.
- Standards guided technical activities, with consideration given to international conventions for large-scale assessments and the scope of SEA-PLM as a first round and as a regional assessment.
- The need for consistency, precision and generalizability of the data collected was recognized by participating countries and stakeholders.
- The Independent Technical Advisory Group of external experts was consulted to adjudicate the final database against the technical standards established for SEA-PLM 2019.
- All countries reached the expected technical standards.