Self-rating and respondent anonymity

Abstract

Background: Most empirical investigations in survey research have been conducted using self-reported or self-evaluated item responses. Such measures are common because they are relatively easy to obtain and are often the only feasible way to assess constructs of interest. In order to improve on the validity of self-reports it has become a common practice to disguise the identities of respondents or to assure them of the confidentiality of their responses.

Purpose: The purpose of this study is to investigate the effect of anonymity on teachers' perceptions of their own teaching skills (i.e. curriculum, pedagogical and assessment skills). Their reporting officers' perceptions of the teachers' teaching skills were used as the reference criterion against which anonymous and identified teachers' perceptions were compared.

Sample: A total of 171 teachers and 39 reporting officers from eight schools in Singapore participated in the study. The teachers were randomly assigned into the two groups of teachers (anonymous and non-anonymous teachers) of equivalent teaching skills.

Design and methods: To ensure validity of comparisons, measures of perceptions must necessarily be made on the same scale for each of the three teaching skills components. Hence Rasch analysis was done using linear Rasch measures in logits rather than the non-linear raw scores. The measures of perceptions obtained from the Rasch analysis for the three groups of respondents were compared using the analysis of variance (ANOVA) and the Least Significance Difference (LSD) test.

Results: The findings indicate that there is no statistically significant difference between the anonymous and non-anonymous teachers' and the reporting officers' perceptions of their teachers' teaching skills.

Conclusions: This study has provided empirical evidence that the effect of anonymity is absent in teachers' perceptions of their own teaching skills, namely curriculum, pedagogical and assessment skills. It was also found that the reporting officers' perceptions of their teachers' teaching skills did not differ significantly from the two teacher groups (anonymous and non-anonymous). It is evident that clarity in roles and responsibilities in the appraisal system may work together with national cultural values (such as Asian abasement and moderation) to explain the teachers' self-rating response behaviours.

Keywords:

• self-rating
• self-evaluation bias
• respondent anonymity
• teaching skills
• Rasch analysis; cultural values