ICCS 2009 OUTCOME MEASURES

Assessment domain(s)
  • Civic education
Achievement and test scales
Scale creation

Item response theory (IRT) modeling – the one-parameter (Rasch) model for dichotomous items as well as the partial credit model for items with more than two categories – was used to scale the test items, with each student respondent being assigned 5 plausible values for the civic knowledge scale.

 


List of achievement scales

Student achievement booklets

International civic knowledge scale (5 plausible values: PV1CIV, PV2CIV, PV3CIV, PV4CIV, PV5CIV)

  • IRT plausible values with a mean of 500 and standard deviation of 100 for equally weighted countries
  • The scaling is based on the 79 adjudicated international cognitive test items and provides internationally comparable results for students’ civic knowledge.
  • All five plausible values should be used for analysis to combine sampling and measurement error.
  • Plausible values were also computed for the small proportion of students for which there was only questionnaire data.
  • Civic knowledge is reported in reference to the following three proficiency levels:
    • Level 3 (563 or higher)
    • Level 2 (479–562)
    • Level 1 (395–478)
  • The descriptions of the competencies of students meeting each of these benchmarks were determined by means of a scale-anchoring process.

 

National civic knowledge scale

  • IRT weighted likelihood estimate (WLE) scores with a mean of 150 and a standard deviation of 10 within each country
  • The scaling is based on the 79 adjudicated international cognitive test items and provides nationally comparable results for students’ civic knowledge.
  • The WLE were computed using the same international parameters; scores are only available for students who participated in the test.

 

CIVED content knowledge scale

  • IRT maximum likelihood estimate (MLE) scores with a mean of 100 and a standard deviation of 20 for equally weighted CIVED countries
  • The scaling is based on the 15 international cognitive link items that pertain to the CIVED subscale measuring students’ civic content knowledge.
  • The MLE were derived using the same item parameters as in CIVED and then transformed to the same scale metric.
  • Scale scores are only available for three out of seven students who responded to the link item cluster and only for those 17 national datasets where the student population is comparable with those surveyed in CIVED in 1999.
  • The data can be analyzed using the same sample weights because booklets within schools were randomly allocated so that the students with CIVED content knowledge scale scores are a random subsample of the selected class.
Questionnaire and background scales
Scale creation

Item response modeling – the one-parameter (Rasch) model for dichotomous items, as well as the partial credit model for items with more than two categories – was typically used to scale questionnaire items.

Cronbach’s alpha coefficient was used as an estimate of the internal consistency of each scale.

For the national index of students’ socioeconomic background (NISB), imputation techniques were implemented.

 

List of background scales

Student questionnaire

Students’ behaviors

  • Students’ discussion of political and social issues outside of school
  • Students’ civic participation in the wider community
  • Students’ civic participation at school

Students’ perceptions of the school context

  • Students’ perceptions of openness in classroom discussions
  • Students’ perceptions of influence on decisions about school
  • Students’ perceptions of student−teacher relations at school
  • Students’ perceptions of the value of participation at school

Students’ democratic value beliefs

  • Students’ support for democratic values

Students’ perceptions of good citizenship

  • Students’ perceptions of the importance of conventional citizenship
  • Students’ perceptions of the importance of social-movement-related citizenship

Students’ civic-related self-beliefs

  • Students’ interest in politics and social issues
  • Students’ sense of internal political efficacy
  • Students’ citizenship self-efficacy

Students’ attitudes toward equal rights

  • Students’ attitudes toward gender equality
  • Students’ attitudes toward equal rights for all ethnic/racial groups
  • Students’ attitudes toward equal rights for immigrants

Students’ attitudes towards institutions and their country

  • Students’ trust in civic institutions
  • Students’ attitudes toward their country

Students’ expected participation in political protest

  • Students’ expected participation in future legal protest
  • Students’ expected adult electoral participation
  • Students’ expected political participation
  • Students’ expected adult electoral participation
  • Students’ expected adult participation in political activities
  • Students’ expected future informal political participation

Students’ attitudes toward the influence of religion in society

  • Students’ attitudes toward the influence of religion in society

 

Student European module questionnaire

  • Students’ sense of European identity
  • Student participation in activities or groups at the European level
  • Student reports on opportunities for learning about Europe at school
  • Student participation in communication about Europe
  • Student attitudes towards European language learning
  • Student attitudes towards freedom of migration within Europe
  • Student attitudes towards restricting migration within Europe
  • Student attitudes towards equal opportunities for other European citizens
  • Student attitudes towards common policies in Europe
  • Student attitudes towards European unification
  • Student attitudes towards a common European currency
  • Student attitudes towards further expansion of the EU
  • Self-reported student knowledge about the EU

 

Student Latin American module questionnaire

  • Students’ sense of Latin American identity
  • Student attitudes towards authoritarianism in government
  • Student attitudes towards corrupt practices in government
  • Student attitudes towards disobeying the law
  • Student attitudes towards neighborhood diversity
  • Student attitudes towards the use of violence
  • Student feelings of empathy towards classmates
  • Students’ personal experience of physical and verbal aggression at school
  • Student reports on frequencies of discussions about civic issues at school

 

Student Asian module questionnaire

  • Student attitudes towards undemocratic government
  • Student attitudes towards obedience to authority
  • Student attitudes towards the preservation of traditional culture
  • Student attitudes towards integrity of the legal system
  • Student attitudes towards corruption in public service
  • Student attitudes towards personal morality of politicians
  • Students’ sense of Asian identity
  • Student perceptions of good citizenship
  • Student attitudes towards the use of connections to obtain public office (guanxi)

 

Teacher questionnaire

Teachers’ perception of school governance

  • Teachers’ participation in school governance
  • Teachers’ perceptions of student influence on decisions about school

Teachers’ perceptions of teaching in classes

  • Confidence in teaching methods
  • Teachers’ use of assessment
  • Teachers’ reports of student participation in class activities

Teachers’ perceptions of participation in the community

  • Teachers’ perceptions of student activities in the community
  • Teachers’ personal participation in activities outside school

Teachers’ perceptions of school and classroom climate

  • Teachers’ perceptions of social problems at school
  • Teachers’ perceptions of student behavior at school
  • Teachers’ perceptions of classroom climate

Teachers’ reports of teaching civic and citizenship education

  • Teachers’ reports on civic and citizenship education activities in class
  • Teachers’ confidence in civic and citizenship education teaching

 

School questionnaire

Principals’ reports on school governance

  • Principals’ perceptions of school autonomy
  • Principals’ perceptions of teacher participation in school governance
  • Principals’ perceptions of parents’ participation in school life
  • Principals’ perceptions of students’ influence on decisions about school

Principals’ reports on the local community

  • Principals’ perceptions of students’ opportunities to participate in community activities
  • Availability of resources in local community
  • Principals’ perceptions of social tension in the community

Principals’ reports on school climate

  • Principals’ perceptions of students’ behavior at school
  • Principals’ perceptions of social problems at school

Principals’ reports on sense of belonging to school

  • Principals’ perceptions of teachers’ sense of belonging to school
  • Principals’ perceptions of students’ sense of belonging to school
  • Principals’ perceptions of non-teaching staff sense of belonging to school