IELS 2025 Design
Methodology
Quantitative Study
Method(s)
Overall approach to data collection
- Proctored assessment of children’s skills
- Self-administered questionnaires for parents and teachers
Target population
All children in centers/schools at the age of five at the time of the assessment
Sample design
Stratified two-stage probability sampling design
First stage: Sampling centers/schools
- Probability proportional to size (PPS), i.e., the expected number of eligible children
- Centers/Schools were randomly sampled from a list of centers/schools that were expected to provide education and care for children of the target age.
- Stratifying schools (optional), could take two forms: explicit or implicit stratification
- Two replacement centers/schools were assigned to each originally selected center/school.
Second stage: Sampling within centers/schools
- Systematic random sampling with equal probabilities
- Children randomly selected from lists of all eligible children within sampled centers/schools
General notes
- Sampling of centers/schools was conducted by the sampling team at IEA Hamburg.
- Sampling procedures within the center/school were carried out by the national study centers, using the Within-school Sampling Software for Windows (WinW3S) provided by the IEA.
Sample size
Intended, per country
- 200 centers/schools
- 15 children in each sampled center/school. In small centres/schools with fewer than 15 children, all children were selected.
- 3,000 children per country, without considering non-response. Therefore, if there were centres/schools with fewer than 15 children, a higher number of centres/schools needed to be selected to maintain the overall number of 3 000 children.
Achieved, across countries
forthcoming
Data collection techniques and instruments
Child assessment
- Children were directly assessed using tablets.
- A study administrator conducted the assessment with each child individually
- 15 minutes per domain
- 4 domains conducted on two assessment days
Questionnaires
- The parent questionnaire was completed either online or on paper and took approximately 20 minutes to complete.
- The staff questionnaire was completed mainly online; in some countries it was administered on paper. Completion took approximately 10 minutes per child, plus an additional 5 minutes for questions about the staff member him-/herself.
Languages
In the main study, the assessment instrument as well as the parent and staff staff questionnaires were administered in the following languages:
- Azerbaijan (AZE)
- Chinese (HCN)
- Dutch (NLD)
- English (ENG, MLT, ARE)
- Flemish (BFL)
- Korean (KOR)
- Maltese (MLT)
- Portuguese (BRA)
- Russian (AZE)
Translation procedures
National versions of all instruments used in the assessment had to be developed through a double-translation-and-reconciliation procedure:
- Source materials for all instruments were developed in English.
- Two independent translators translated the source material into the target language.
- A third person merged these two versions into a single national version.
- The translated/adapted versions of the direct assessments and the questionnaires were submitted to cApStAn for translation verification.
Quality control of operations
Measures during data collection
- National project managers (NPMs) in each participating country responsible for data collection
- Standardized survey operation procedures: step-by-step documentation of all operational activities provided with manuals
- Full-scale field test of all instruments and operational procedures (in each participating country)
- Appointment of a national data manager (NDM) to oversee and implement all data tasks
- National Quality Assurance Monitoring Program
- International Quality Assurance Monitoring Program
- Training of NPMs, center/school coordinators, study administrators, national quality assurance monitors (NQAMs)
- Provision of software tools for supporting activities
- Survey activities questionnaire (SAQ) to be completed by NPMs
- Double data entry for questionnaires
- Visits to centers/schools by international quality assurance monitors (IQAMs)
Measures during data processing and cleaning
- Thorough testing of all data cleaning programs with simulated data sets
- Material receipt database
- National adaptation database
- Standardized, iterative four-step cleaning process
- Documentation and structure check
- Identification variable (ID) and linkage cleaning
- Background cleaning (resolving inconsistencies in questionnaire data)
- Valid range checks
- Repetition of data cleaning until all data shown to be consistent and comparable
- Identification of irregularities in data patterns and correction of data errors
Other sources