Are questionnaire scales which measure non-cognitive constructs suitable as school effectiveness criteria? A measurement invariance analysis

References

• Asparouhov, T., & Muthén, B. (2014). Multiple-group factor analysis alignment. Structural Equation Modeling: A Multidisciplinary Journal, 21(4), 495–508. https://doi.org/https://doi.org/10.1080/10705511.2014.919210
   Web of Science ®Google Scholar
• Baumgartner, H., & Steenkamp, J.-B. E. M. (1998). Multi-group latent variable models for varying numbers of items and factors with cross-national and longitudinal applications. Marketing Letters, 9(1), 21–35. https://doi.org/https://doi.org/10.1023/A:1007911903032
   Google Scholar
• Bos, W., Bonsen, M., & Gröhlich, C. (Eds.). (2009). KESS 7 – Kompetenzen und Einstellungen von Schülerinnen und Schülern an Hamburger Schulen zu Beginn der Jahrgangsstufe 7 [KESS 7 – Competencies and attitudes of students at Hamburg schools at the beginning of Grade 7]. Waxmann.
   Google Scholar
• Bos, W., & Gröhlich, C. (Eds.). (2010). KESS 8: Kompetenzen und Einstellungen von Schülerinnen und Schülern am Ende der Jahrgangsstufe 8 [KESS 8: Competencies and attitudes of students at the end of Grade 8]. Waxmann.
   Google Scholar
• Bos, W., & Pietsch, M. (Eds.). (2006). KESS 4 – Kompetenzen und Einstellungen von Schülerinnen und Schülern am Ende der Jahrgangsstufe 4 in Hamburger Grundschulen [KESS 4 – Competencies and attitudes of students at the end of Grade 4 in Hamburg primary schools]. Waxmann.
   Google Scholar
• Brown, T. A. (2006). Confirmatory factor analysis for applied research. Guilford Press.
   Google Scholar
• Byrne, B. M., Shavelson, R. J., & Muthén, B. (1989). Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. Psychological Bulletin, 105(3), 456–466. https://doi.org/https://doi.org/10.1037/0033-2909.105.3.456
   Web of Science ®Google Scholar
• Chen, F. F. (2008). What happens if we compare chopsticks with forks? The impact of making inappropriate comparisons in cross-cultural research. Journal of Personality and Social Psychology, 95(5), 1005–1018. https://doi.org/https://doi.org/10.1037/a0013193
   PubMed Web of Science ®Google Scholar
• Cheung, G. W., & Rensvold, R. B. (1998). Cross-cultural comparisons using non-invariant measurement items. Applied Behavioral Science Review, 6(1), 93–110. https://doi.org/https://doi.org/10.1016/S1068-8595(99)80006-3
   Google Scholar
• De Maeyer, S., van den Bergh, H., Rymenans, R., Van Petegem, P., & Rijlaarsdam, G. (2010). Effectiveness criteria in school effectiveness studies: Further research on the choice for a multivariate model. Educational Research Review, 5(1), 81–96. https://doi.org/https://doi.org/10.1016/j.edurev.2009.09.001
   Web of Science ®Google Scholar
• Fend, H. (2008). Neue Theorie der Schule [New theory of school] (2nd rev. ed.). VS Verlag für Sozialwissenschaften. https://doi.org/https://doi.org/10.1007/978-3-531-91788-7
   Google Scholar
• Guenole, N., & Brown, A. (2014). The consequences of ignoring measurement invariance for path coefficients in structural equation models. Frontiers in Psychology, 5: 980. https://doi.org/https://doi.org/10.3389/fpsyg.2014.00980
   PubMed Web of Science ®Google Scholar
• Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. https://doi.org/https://doi.org/10.1080/10705519909540118
   Web of Science ®Google Scholar
• Jerusalem, M. (1984). Selbstbezogene Kognitionen in schulischen Bezugsgruppen: Eine Längsschnittstudie [Self-related cognitions in scholastic reference groups: A longitudinal study]. Free University Berlin, Department of Psychology.
   Google Scholar
• Jopt, U.-J. (1978). Selbstkonzept und Ursachenerklärung in der Schule: Zur Attribuierung von Schulleistungen [Self-concept and attribution in school: Attribution of school performance]. Kamp.
   Google Scholar
• Kelly, D. L. (2003). Developing the PIRLS background questionnaires. In M. O. Martin, I. V. S. Mullis, & A. M. Kennedy (Eds.), PIRLS 2001 technical report (pp. 29–39). TIMSS & PIRLS International Study Center, Lynch School of Education, Boston College.
   Google Scholar
• Kim, E. S., Cao, C., Wang, Y., & Nguyen, D. T. (2017). Measurement invariance testing with many groups: A comparison of five approaches. Structural Equation Modeling: A Multidisciplinary Journal, 24(4), 524–544. https://doi.org/https://doi.org/10.1080/10705511.2017.1304822
   Web of Science ®Google Scholar
• Krapp, A., & Prenzel, M. (2011). Research on interest in science: Theories, methods, and findings. International Journal of Science Education, 33(1), 27–50. https://doi.org/https://doi.org/10.1080/09500693.2010.518645
   Web of Science ®Google Scholar
• Kuhl, J. (1984). Tatsächliche und phänomenale Hilflosigkeit: Vermittlung von Leistungsdefiziten nach massiver Mißerfolgsinduktion [Actual and phenomenal helplessness: Mediation of performance deficits after massive failure induction]. In F. E. Weinert & R. H. Kluwe (Eds.), Metakognition, Motivation und Lernen (pp. 192–209). Kohlhammer.
   Google Scholar
• Kuhl, J. (1998). Wille und Persönlichkeit: Funktionsanalyse der Selbststeuerung [Will and personality: Functional analysis of self-control]. Psychologische Rundschau, 49(2), 61–77.
   Google Scholar
• Little, T. D., Preacher, K. J., Selig, J. P., & Card, N. A. (2007). New developments in latent variable panel analyses of longitudinal data. International Journal of Behavioral Development, 31(4), 357–365. https://doi.org/https://doi.org/10.1177/0165025407077757
   Web of Science ®Google Scholar
• Lüdtke, O., Robitzsch, A., Trautwein, U., & Köller, O. (2007). Umgang mit fehlenden Werten in der psychologischen Forschung [Dealing with missing values in psychological research]. Psychologische Rundschau, 58(2), 103–117. https://doi.org/https://doi.org/10.1026/0033-3042.58.2.103
   Web of Science ®Google Scholar
• Lüdtke, O., Trautwein, U., Schnyder, I., & Niggli, A. (2007). Simultane Analysen auf Schüler- und Klassenebene [Simultaneous analyses at student and class level]. Zeitschrift für Entwicklungspsychologie und Pädagogische Psychologie, 39(1), 1–11. https://doi.org/https://doi.org/10.1026/0049-8637.39.1.1
   Web of Science ®Google Scholar
• Marsh, H. W. (1990). A multidimensional, hierarchical model of self-concept: Theoretical and empirical justification. Educational Psychology Review, 2(2), 77–172. https://doi.org/https://doi.org/10.1007/bf01322177
   Web of Science ®Google Scholar
• Marsh, H. W., Guo, J., Parker, P. D., Nagengast, B., Asparouhov, T., Muthen, B., & Dicke, T. (2018). What to do when scalar invariance fails: The extended alignment method for multi-group factor analysis comparison of latent means across many groups. Psychological Methods, 23(3), 524–545. https://doi.org/https://doi.org/10.1037/met0000113
   PubMed Web of Science ®Google Scholar
• Möller, J., & Bonerad, E.-M. (2007). Fragebogen zur habituellen Lesemotivation [Questionnaire on habitual reading motivation]. Psychologie in Erziehung und Unterricht, 4, 259–267.
   Google Scholar
• Möller, J., & Schiefele, U. (2004). Motivationale Grundlagen der Lesekompetenz [Motivational foundations of reading literacy]. In U. Schiefele, C. Artelt, W. Schneider, & P. Stanat (Eds.), Struktur, Entwicklung und Förderung von Lesekompetenz: Vertiefende Analysen im Rahmen von PISA 2000 (pp. 101–124). VS Verlag für Sozialwissenschaften.
   Google Scholar
• Mullis, I. V. S., Martin, M. O., Gonzalez, E. J., & Kennedy, A. M. (2003). PIRLS 2001 international report. IEA ‘s study of reading literacy achievement in primary school in 35 countries. International Study Centre, Lynch School of Education, Boston College.
   Google Scholar
• Mullis, I. V. S., Martin, M. O., Kennedy, A. M., Trong, K. L., & Sainsbury, M. (Eds.). (2009). PIRLS 2011 assessment framework. TIMSS & PIRLS International Study Center, Lynch School of Education, Boston College.
   Google Scholar
• Muthén, B., & Asparouhov, T. (2013). New methods for the study of measurement invariance with many groups [Technical report]. https://www.statmodel.com/download/PolAn.pdf
   Google Scholar
• Putnick, D. L., & Bornstein, M. H. (2016). Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Developmental Review, 41, 71–90. https://doi.org/https://doi.org/10.1016/j.dr.2016.06.004
   PubMed Web of Science ®Google Scholar
• Reynolds, D., & Teddlie, C. (2000). The future agenda for school effectiveness research. In C. Teddlie & D. Reynolds (Eds.), The international handbook of school effectiveness research (pp. 322–343). Falmer Press/Routledge.
   Google Scholar
• Rutkowski, L., & Svetina, D. (2014). Assessing the hypothesis of measurement invariance in the context of large-scale international surveys. Educational and Psychological Measurement, 74(1), 31–57. https://doi.org/https://doi.org/10.1177/0013164413498257
   Web of Science ®Google Scholar
• Satorra, A., & Bentler, P. M. (1994). Corrections to test statistics and standard errors in covariance structure analysis. In A. von Eye & C. C. Clogg (Eds.), Latent variables analysis: Applications for developmental research (pp. 399–419). Sage.
   Google Scholar
• Seidel, T. (2008). Stichwort: Schuleffektivitätskriterien in der internationalen empirischen Forschung [Keyword: School effectiveness criteria in international empirical research]. Zeitschrift für Erziehungswissenschaft, 11(3), 348–367. https://doi.org/https://doi.org/10.1007/s11618-008-0032-4
   Web of Science ®Google Scholar
• Steinmetz, H. (2013). Analyzing observed composite differences across groups. Methodology, 9(1), 1–12. https://doi.org/https://doi.org/10.1027/1614-2241/a000049
   Web of Science ®Google Scholar
• Trautwein, U. (2003). Schule und Selbstwert [School and self-worth]. Waxmann.
   Google Scholar
• Turner, J. C., & Patrick, H. (2004). Motivational influences on student participation in classroom learning activities. Teachers College Record, 106(9), 1759–1785.
   Web of Science ®Google Scholar
• Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4–70. https://doi.org/https://doi.org/10.1177/109442810031002
   Web of Science ®Google Scholar
• van der Wal, M., & Waslander, S. (2007). Traditional and non-traditional educational outcomes: Trade-off or complementarity? School Effectiveness and School Improvement, 18(4), 409–428. https://doi.org/https://doi.org/10.1080/09243450701712502
   Web of Science ®Google Scholar
• van de Vijver, F. J. R., Avvisati, F., Davidov, E., Eid, M., Fox, J.-P., Le Donné, N., Lek, K., Meuleman, B., Paccagnella, M., & van de Schoot, R. (2019). Invariance analyses in large-scale studies (OECD Working Paper No. 201). Organisation for Economic Co-operation and Development. https://doi.org/https://doi.org/10.1787/254738dd-en
   Google Scholar
• van de Vijver, F. J. R., & Poortinga, Y. H. (2002). Structural equivalence in multilevel research. Journal of Cross-Cultural Psychology, 33(2), 141–156. https://doi.org/https://doi.org/10.1177/0022022102033002002
   Web of Science ®Google Scholar
• Vieluf, U., Ivanov, S., & Nikolova, R. (Eds.). (2011). KESS 10/11– Kompetenzen und Einstellungen von Schülerinnen und Schülern an Hamburger Schulen am Ende der Sekundarstufe I und zu Beginn der gymnasialen Oberstufe [KESS 10/11 – Competencies and attitudes of pupils at Hamburg schools at the end of lower secondary school and at the beginning of upper secondary school]. Waxmann.
   Google Scholar
• Vieluf, U., Ivanov, S., & Nikolova, R. (2014). KESS 12/13: Kompetenzen und Einstellungen von Schülerinnen und Schülern an Hamburger Schulen am Ende der gymnasialen Oberstufe [KESS 12/13: Competencies and attitudes of pupils at Hamburg schools at the end of the upper secondary school]. https://bildungsserver.hamburg.de/contentblob/4396048/6b49c68061321ae400aaa4f7250ebe9f/data/kess12-13.pdf
   Google Scholar
• Walther, G., Geiser, H., Langeheine, R., & Lobemeier, K. (2003). Mathematische Kompetenzen am Ende der vierten Jahrgangsstufe [Mathematical competencies at the end of fourth grade]. In W. Bos, E.-M. Lankes, M. Prenzel, K. Schwippert, G. Walther, & R. Valtin (Eds.), Erste Ergebnisse aus IGLU. Schülerleistungen am Ende der vierten Jahrgangsstufe im internationalen Vergleich (pp. 189–226). Waxmann.
   Google Scholar