The conceptualisation implies the statistical model: implications for measuring domains of teaching quality

References

• Allen, J. P., Gregory, A., Mikami, A., Lun, J., Hamre, B. K., & Pianta, R. C. (2013). Observations of effective teacher-student interactions in secondary school classrooms: predicting student achievement with the classroom assessment scoring system–secondary. School Psychology Review, 42(1), 76–97. https://doi.org/10.1080/02796015.2013.12087492
   PubMed Web of Science ®Google Scholar
• Andrich, D. (2004). Controversy and the rasch model: A characteristic of incompatible paradigms? Medical Care, 42(1), 7. https://doi.org/10.1097/01.mlr.0000103528.48582.7c
   Web of Science ®Google Scholar
• Bagozzi, R. P. (2007). On the meaning of formative measurement and how it differs from reflective measurement: Comment on Howell, Breivik, and Wilcox. Psychological Methods, 12(2), 229–337. https://doi.org/10.1037/1082-989X.12.2.229
   PubMed Web of Science ®Google Scholar
• Bell, C. A., Dobbelaer, M. J., Klette, K., & Visscher, A. (2019). Qualities of classroom observation systems. School Effectiveness and School Improvement, 30(1), 1–27. https://doi.org/10.1080/09243453.2018.1539014
   Web of Science ®Google Scholar
• Berlin, R., & Cohen, J. (2020). The convergence of emotionally supportive learning environments and college and career ready mathematical engagement in upper elementary classrooms. American Educational Research Association Open, 6(3), 6(3. https://doi.org/10.1177/2332858420957612
   Google Scholar
• Blazar, D., Braslow, D., Charalambous, C. Y., & Hill, H. C. (2017). Attending to general and mathematics-specific dimensions of teaching: exploring factors across two observation instruments. Educational Assessment, 22(2), 71–94. https://doi.org/10.1080/10627197.2017.1309274
   Web of Science ®Google Scholar
• Bollen, K. A., & Diamantopoulos, A. (2017). In defense of causal-formative indicators: A minority report. Psychological Methods, 22(3), 581–596. https://doi.org/10.1037/met0000056
   PubMed Web of Science ®Google Scholar
• Bollen, K., & Lennox, R. (1991). Conventional wisdom on measurement: A structural equation perspective. Psychological Bulletin, 110(2), 305–314. https://doi.org/10.1037/0033-2909.110.2.305
   Web of Science ®Google Scholar
• Camacho, D. A., Moore, S. A., Pas, E. T., & Bradshaw, C. P. (2022). Interactional quality in middle schools: Latent profiles and their associations with teacher, classroom, and school compositional factors. Journal of School Psychology, 93, 79–97. https://doi.org/10.1016/j.jsp.2022.06.001
   PubMed Web of Science ®Google Scholar
• Charalambous, C. Y., Praetorius, A.-K., Sammons, P., Walkowiak, T., Jentsch, A., & Kyriakides, L. (2021). Working more collaboratively to better understand teaching and its quality: Challenges faced and possible solutions. Studies in Educational Evaluation, 71, 101092. https://doi.org/10.1016/j.stueduc.2021.101092
   Web of Science ®Google Scholar
• Chernyshenko, O. S., Stark, S., Chan, K. Y., Drasgow, F., & Williams, B. (2001). Fitting item response theory models to two personality inventories: Issues and insights. Multivariate Behavioral Research, 36(4), 523–562. https://doi.org/10.1207/S15327906MBR3604_03
   PubMed Web of Science ®Google Scholar
• Collins, L. M., & Lanza, S. T. (2009). Latent class and latent transition analysis: With applications in the social, behavioral, and health sciences (Vol. 718). John Wiley & Sons.
   Google Scholar
• De Boeck, P., Bakker, M., Zwitser, R., Nivard, M., Hofman, A., Tuerlinckx, F., & Partchev, I. (2011). The estimation of item response models with the lmer function from the lme4 package in R. Journal of Statistical Software, 39(12), 1–28. https://doi.org/10.18637/jss.v039.i12
   PubMed Web of Science ®Google Scholar
• Dobbelaer, M. J. (2019). The quality and qualities of classroom observation systems. University of Twente. https://research.utwente.nl/en/publications/the-quality-and-qualities-of-classroom-observation-systems
   Google Scholar
• Downer, J. T., López, M. L., Grimm, K. J., Hamagami, A., Pianta, R. C., & Howes, C. (2012). Observations of teacher-child interactions in classrooms serving latinos and dual language learners: applicability of the classroom assessment scoring system in diverse settings. Early Childhood Research Quarterly, 27(1), 21–32. https://doi.org/10.1016/j.ecresq.2011.07.005
   Web of Science ®Google Scholar
• Edelsbrunner, P. A. (2022). A model and its fit lie in the eye of the beholder: Long live the sum score. Frontiers in Psychology, 13, 986767. https://doi.org/10.3389/fpsyg.2022.986767
   PubMed Web of Science ®Google Scholar
• Edelsbrunner, P. A., & Dablander, F. (2019). The psychometric modelling of scientific reasoning: A review and recommendations for future avenues. Educational Psychology Review, 31(1), 1–34. https://doi.org/10.1007/s10648-018-9455-5
   Web of Science ®Google Scholar
• Edwards, J. R., & Bagozzi, R. P. (2000). On the nature and direction of relationships between constructs and measures. Psychological Methods, 5(2), 155. https://doi.org/10.1037/1082-989X.5.2.155
   PubMed Web of Science ®Google Scholar
• Ellis, J. L., & Junker, B. W. (1997). Tail-measurability in monotone latent variable models. Psychometrika, 62(4), 495–523. https://doi.org/10.1007/BF02294640
   Web of Science ®Google Scholar
• Engelhard, G., Jr. (2013). Invariant measurement: Using Rasch models in the social, behavioral, and health sciences. Routledge.
   Google Scholar
• Finch, W. H., & Bronk, K. C. (2011). Conducting confirmatory latent class analysis using M plus. Structural Equation Modelling, 18(1), 132–151. https://doi.org/10.1080/10705511.2011.532732
   Web of Science ®Google Scholar
• Fiske, D. W., & Butler, J. M. (1963). The experimental conditions for measuring individual differences. Educational and Psychological Measurement, 23(2), 249–266. https://doi.org/10.1177/001316446302300203
   Web of Science ®Google Scholar
• Fried, E. I. (2020). Lack of theory building and testing impedes progress in the factor and network literature. Psychological Inquiry, 31(4), 271–288. https://doi.org/10.1080/1047840X.2020.1853461
   Web of Science ®Google Scholar
• Garrett, R., & Steinberg, M. P. (2015). Examining teacher effectiveness using classroom observation scores evidence from the randomization of teachers to students. Educational Evaluation and Policy Analysis, 37(2), 224–242. https://doi.org/10.3102/0162373714537551
   Web of Science ®Google Scholar
• Gruijters, S. L. K., Fleuren, B. P. I., & Peters, G.-J. Y. (2021). Crossing the seven Cs of internal consistency: Assessing the reliability of formative instruments. [Preprint] PsyArXiv. https://doi.org/10.31234/osf.io/qar39
   Google Scholar
• Hafen, C. A., Hamre, B. K., Allen, J. P., Bell, C. A., Gitomer, D. H., & Pianta, R. C. (2014). Teaching through interactions in secondary school classrooms revisiting the factor structure and practical application of the classroom assessment scoring system–secondary. The Journal of Early Adolescence, 35(5–6), 651–680. https://doi.org/10.1177/0272431614537117
   PubMed Web of Science ®Google Scholar
• Halpin, P. F., & Kieffer, M. J. (2015). Describing profiles of instructional practice a new approach to analyzing classroom observation data. Educational Researcher, 44(5), 263–277. https://doi.org/10.3102/0013189X15590804
   Web of Science ®Google Scholar
• Hamre, B. K., Hatfield, B., Pianta, R. C., & Jamil, F. (2014). Evidence for general and domain-specific elements of teacher–child interactions: associations with preschool children’s development. Child Development, 85(3), 1257–1274. https://doi.org/10.1111/cdev.12184
   PubMed Web of Science ®Google Scholar
• Hamre, B. K., Pianta, R. C., Downer, J. T., DeCoster, J., Mashburn, A., Jones, S. M., Brown, J. L., Cappella, E., Atkins, M., Rivers, S. E., Brackett, M., & Hamagami, A. (2013). Teaching through interactions: Testing a developmental framework of teacher effectiveness in over 4,000 classrooms. The Elementary School Journal, 113(4), 461–487. https://doi.org/10.1086/669616
   PubMed Web of Science ®Google Scholar
• Harvey, R. J. (2016). Improving measurement via item response theory: Great idea, but hold the rasch. The Counseling Psychologist, 44(2), 195–204. https://doi.org/10.1177/0011000015615427
   Web of Science ®Google Scholar
• Hershberger, S. L. (1994). The specification of equivalent models before the collection of data. In A. von Eye & C. C. Clogg (Eds.), Latent variables analysis: Applications for developmental research (pp. 68–105). Sage Publications, Inc.
   Google Scholar
• Hickendorff, M., Edelsbrunner, P. A., McMullen, J., Schneider, M., & Trezise, K. (2018). Informative tools for characterizing individual differences in learning: Latent class, latent profile, and latent transition analysis. Learning and Individual Differences, 66, 4–15. https://doi.org/10.1016/j.lindif.2017.11.001
   Web of Science ®Google Scholar
• Hill, H. C., Charalambous, C. Y., & Kraft, M. A. (2012). When rater reliability is not enough teacher observation systems and a case for the generalizability study. Educational Researcher, 41(2), 56–64. https://doi.org/10.3102/0013189X12437203
   Web of Science ®Google Scholar
• Hoelscher, M. (2017). “It’s the method, stupid.” interrelations between methodological and theoretical advances: The example of comparing higher education systems internationally. Research in Comparative and International Education, 12(3), 347–359. https://doi.org/10.1177/1745499917724287
   Web of Science ®Google Scholar
• Holmes, L., & Schumacker, R. (2020). Latent class analysis of teacher characteristics: Can we identify effective teachers? Measurement: Interdisciplinary Research and Perspectives, 18(2), 75–86. https://doi.org/10.1080/15366367.2019.1657350
   Web of Science ®Google Scholar
• Howell, R. D., Breivik, E., & Wilcox, J. B. (2013). Formative measurement: A critical perspective. ACM SIGMIS Database: The DATABASE for Advances in Information Systems, 44(4), 44–55. https://doi.org/10.1145/2544415.2544418
   Google Scholar
• Hubbard, R., Haig, B. D., & Parsa, R. A. (2019). The limited role of formal statistical inference in scientific inference. The American Statistician, 73(sup1), 91–98. https://doi.org/10.1080/00031305.2018.1464947
   Web of Science ®Google Scholar
• Hu, B. Y., Fan, X., Gu, C., & Yang, N. (2016). Applicability of the classroom assessment scoring system in Chinese preschools based on psychometric evidence. Early Education and Development, 27(5), 714–734. https://doi.org/10.1080/10409289.2016.1113069
   Web of Science ®Google Scholar
• Jarvis, C. B., MacKenzie, S. B., & Podsakoff, P. M. (2003). A critical review of construct indicators and measurement model misspecification in marketing and consumer research. Journal of Consumer Research, 30(2), 199–218. https://doi.org/10.1086/376806
   Web of Science ®Google Scholar
• Johnson, G. J., Bruner, G. C., II, & Kumar, A. (2006). Interactivity and its facets revisited: Theory and empirical test. Journal of Advertising, 35(4), 35–52. https://doi.org/10.2753/JOA0091-3367350403
   Web of Science ®Google Scholar
• Jones, E., & Bergin, C. (2019). Evaluating teacher effectiveness using classroom observations: A rasch analysis of the rater effects of principals. Educational Assessment, 24(2), 91–118. https://doi.org/10.1080/10627197.2018.1564272
   Web of Science ®Google Scholar
• Kane, T. J., Taylor, E. S., Tyler, J. H., & Wooten, A. L. (2011). Evaluating teacher effectiveness. Education Next, 11(3), 54–61. https://www.educationnext.org/evaluating-teacher-effectiveness/
   Google Scholar
• Kim, J.-O., & Mueller, C. (1978). Introduction to factor analysis. SAGE Publications, Inc. https://doi.org/10.4135/9781412984652
   Google Scholar
• Kline, R. B. (2023a). Assumptions in structural equation modeling. In R. H. Hoyle (Ed.), Handbook of structural equation modeling (2nd ed., pp. 128–144). The Guilford Press.
   Google Scholar
• Kline, R. B. (2023b). Principles and practice of structural equation modeling. Guilford publications.
   Google Scholar
• Lewis-Beck, M., Bryman, A., & Futing Liao, T. (2004). Confirmatory factor analysis. In The SAGE encyclopedia of social science research methods. Sage Publications, Inc.
   Google Scholar
• Lin, C. H., Sher, P. J., & Shih, H. Y. (2005). Past progress and future directions in conceptualizing customer perceived value. International Journal of Service Industry Management, 16(4), 318–336. https://doi.org/10.1108/09564230510613988
   Google Scholar
• Little, T. D. (2013). Longitudinal structural equation modelling. The Guilford Press.
   Google Scholar
• Lubke, G., & Neale, M. C. (2006). Distinguishing between latent classes and continuous factors: Resolution by maximum likelihood? Multivariate Behavioral Research, 41(4), 499–532. https://doi.org/10.1207/s15327906mbr4104_4
   PubMed Web of Science ®Google Scholar
• MacKenzie, S. B., Podsakoff, P. M., & Jarvis, C. B. (2005). The problem of measurement model misspecification in behavioral and organizational research and some recommended solutions. Journal of Applied Psychology, 90(4), 710–730. https://doi.org/10.1037/0021-9010.90.4.710
   PubMed Web of Science ®Google Scholar
• Markus, K. A., & Borsboom, D. (2013). Frontiers of test validity theory: Measurement, causation, and meaning. Routledge.
   Google Scholar
• Masyn, K. E. (2013). Latent class analysis and finite mixture modelling. In T. D. Little (Ed.), The Oxford handbook of quantitative methods (pp. 551–611). Oxford University Press.
   Google Scholar
• McCaffrey, D. F., Yuan, K., Savitsky, T. D., Lockwood, J. R., & Edelen, M. O. (2014). Uncovering multivariate structure in classroom observations in the presence of rater errors. Educational Measurement Issues & Practice, 34(2), 34–46. https://doi.org/10.1111/emip.12061
   Google Scholar
• McDonald, R. P. (1985). Factor analysis and related methods. Lawrence Erlbaum Associates.
   Google Scholar
• McDonald, R. P. (2013). Test theory: A unified treatment. Psychology Press.
   Google Scholar
• McGrane, J. A., & Maul, A. (2020). The human sciences, models and metrological mythology. Measurement, 152, 107346. https://doi.org/10.1016/j.measurement.2019.107346
   Web of Science ®Google Scholar
• Meijer, R. R., & Tendeiro, J. N. (2018). Unidimensional item response theory. In P. Irwing, T. Booth, & D. J. Hugh (Eds.), The Wiley handbook of psychometric testing: A multidisciplinary reference on survey, scale and test development (pp. 413–433). Wiley. https://doi.org/10.1002/9781118489772.ch15
   Google Scholar
• Molenaar, P. C. M., & von Eye, A. (1994). On the arbitrary nature of latent variables. In A. von Eye & C. C. Clogg (Eds.), Latent variables analysis: Applications for developmental research (pp. 226–242). Sage Publications, Inc.
   Google Scholar
• OECD. (2020). Global teaching insights: a video study of teaching. OECD Publishing. https://doi.org/10.1787/20d6f36b-en
   Google Scholar
• Penttinen, V., Pakarinen, E., von Suchodoletz, A., & Lerkkanen, M. K. (2022). The quality of teacher-child interactions and teachers’ occupational well-being in Finnish kindergartens: A person-centered approach. Early Education and Development, 34(7), 1–22. https://doi.org/10.1080/10409289.2022.2139549
   Web of Science ®Google Scholar
• Peterson, C. H., Gischlar, K. L., & Peterson, N. A. (2017). Item construction using reflective, formative, or rasch measurement models: Implications for group work. The Journal for Specialists in Group Work, 42(1), 17–32. https://doi.org/10.1080/01933922.2016.1264523
   Web of Science ®Google Scholar
• Rademaker, M. E. (2020). Composite-based structural equation modelling [ Doctoral dissertation]. Universität Würzburg.
   Google Scholar
• Raykov, T., & Marcoulides, G. A. (2011). Introduction to psychometric theory. Routledge. https://doi.org/10.4324/9780203841624
   Google Scholar
• Reise, S. P., Widaman, K. F., & Pugh, R. H. (1993). Confirmatory factor analysis and item response theory: Two approaches for exploring measurement invariance. Psychological Bulletin, 114(3), 552. https://doi.org/10.1037/0033-2909.114.3.552
   PubMed Web of Science ®Google Scholar
• Rhemtulla, M., van Bork, R., & Borsboom, D. (2020). Worse than measurement error: Consequences of inappropriate latent variable measurement models. Psychological Methods, 25(1), 30–45. https://doi.org/10.1037/met0000220
   PubMed Web of Science ®Google Scholar
• Ruiz, D. M., Gremler, D. D., Washburn, J. H., & Carrión, G. C. (2008). Service value revisited: Specifying a higher-order, formative measure. Journal of Business Research, 61(12), 1278–1291. https://doi.org/10.1016/j.jbusres.2008.01.015
   Web of Science ®Google Scholar
• Salminen, J., Lerkkanen, M.-K., Poikkeus, A.-M., Pakarinen, E., Siekkinen, M., Hännikäinen, M., Poikonen, P.-L., & Rasku-Puttonen, H. (2012). Observed classroom quality profiles of kindergarten classrooms in Finland. Early Education and Development, 23(5), 654–677. https://doi.org/10.1080/10409289.2011.574267
   Web of Science ®Google Scholar
• Sarstedt, M., Hair, J. F., Ringle, C. M., Thiele, K. O., & Gudergan, S. P. (2016). Estimation issues with PLS and CBSEM: Where the bias lies! Journal of Business Research, 69(10), 3998–4010. https://doi.org/10.1016/j.jbusres.2016.06.007
   Web of Science ®Google Scholar
• Schmiege, S. J., Masyn, K. E., & Bryan, A. D. (2018). Confirmatory latent class analysis: Illustrations of empirically driven and theoretically driven model constraints. Organizational Research Methods, 21(4), 983–1001. https://doi.org/10.1177/1094428117747689
   Web of Science ®Google Scholar
• Schuberth, F. (2021a). Confirmatory composite analysis using partial least squares: Setting the record straight. Review of Managerial Science, 15(5), 1311–1345. https://doi.org/10.1007/s11846-020-00405-0
   Web of Science ®Google Scholar
• Schuberth, F. (2021b). The Henseler-Ogasawara specification of composites in structural equation modeling: A tutorial. Psychological Methods, (4), 843–859. https://doi.org/10.1037/met0000432
   PubMed Web of Science ®Google Scholar
• Schweig, J. D., & Martínez, J. F. (2021). Understanding (Dis)agreement in student ratings of teaching and the quality of the learning environment. In W. Rollett, H. Bijlsma, & S. Röhl (Eds.), Student feedback on teaching in schools: using student perceptions for the development of teaching and teachers (pp. 91–110). Springer International Publishing. https://doi.org/10.1007/978-3-030-75150-0_6
   Google Scholar
• Simonetto, A. (2012). Formative and reflective models to determine latent construct [conference presentation]. 46th Scientific Meeting of the Italian Statistical Society, Rome, Italy.
   Google Scholar
• Stadler, M., Sailer, M., & Fischer, F. (2021). Knowledge as a formative construct: A good alpha is not always better. New Ideas in Psychology, 60, 100832. https://doi.org/10.1016/j.newideapsych.2020.100832
   Web of Science ®Google Scholar
• Stapleton, L. M., Yang, J. S., & Hancock, G. R. (2016). Construct meaning in multilevel settings. Journal of Educational and Behavioral Statistics, 41(5), 481–520. https://doi.org/10.3102/1076998616646200
   Web of Science ®Google Scholar
• Taber, K. S. (2018). The use of Cronbach’s alpha when developing and reporting research instruments in science education. Research in Science Education, 48(6), 1273–1296. https://doi.org/10.1007/s11165-016-9602-2
   Web of Science ®Google Scholar
• Taut, S., & Rakoczy, K. (2016). Observing instructional quality in the context of school evaluation. Learning and Instruction, 46, 45–60. https://doi.org/10.1016/j.learninstruc.2016.08.003
   Web of Science ®Google Scholar
• van de Grift, W. J. C. M., Helms-Lorenz, M., & Maulana, R. (2014). Teaching skills of student teachers: Calibration of an evaluation instrument and its value in predicting student academic engagement. Studies in Educational Evaluation, 43, 150–159. https://doi.org/10.1016/j.stueduc.2014.09.003
   Web of Science ®Google Scholar
• van de Grift, W. J. C. M., Houtveen, T. A. M., Hurk, H. T. G. V. D., & Terpstra, O. (2019). Measuring teaching skills in elementary education using the rasch model. School Effectiveness and School Improvement, 30(4), 455–486. https://doi.org/10.1080/09243453.2019.1577743
   Web of Science ®Google Scholar
• van der Lans, R. M., van de Grift, W. J. C. M., & Veen, K. V. (2018). Developing an instrument for teacher feedback: Using the rasch model to explore teachers’ development of effective teaching strategies and behaviors. The Journal of Experimental Education, 86(2), 247–264. https://doi.org/10.1080/00220973.2016.1268086
   Web of Science ®Google Scholar
• VanderWeele, T. J. (2022). Constructed measures and causal inference: Towards a new model of measurement for psychosocial constructs. Epidemiology (Cambridge, Mass), 33(1), 141. https://doi.org/10.1097/EDE.0000000000001434
   PubMed Web of Science ®Google Scholar
• Vermunt, J., & Magidson, J. (2004). Latent class analysis. In The Sage encyclopedia of social science research methods (pp. 549–553). SAGE Publications, Inc.
   Google Scholar
• Virtanen, T. E., Pakarinen, E., Lerkkanen, M.-K., Poikkeus, A.-M., Siekkinen, M., & Nurmi, J.-E. (2018). A validation study of classroom assessment scoring system–secondary in the Finnish school context. The Journal of Early Adolescence, 38(6), 849–880. https://doi.org/10.1177/0272431617699944
   Web of Science ®Google Scholar
• Virtanen, T. E., Vaaland, G. S., & Ertesvåg, S. K. (2019). Associations between observed patterns of classroom interactions and teacher wellbeing in lower secondary school. Teaching and Teacher Education, 77, 240–252. https://doi.org/10.1016/j.tate.2018.10.013
   Web of Science ®Google Scholar
• White, M. (2017). Generalizability of scores from classroom observation instruments. University of Michigan. http://hdl.handle.net/2027.42/138742
   Google Scholar
• White, M., & Klette, K. (2023). What’s in a score? Problematizing interpretations of observation scores. Studies in Educational Evaluation, 77, 101238. https://doi.org/10.1016/j.stueduc.2023.101238
   Web of Science ®Google Scholar
• White, M., Luoto, J. M., Klette, K., & Blikstad Balas, M. (2022, August 19). Bringing the theory and measurement of teaching into alignment Studies in Educational Evaluation, 75. https://doi.org/10.1016/j.stueduc.2022.101204
   Web of Science ®Google Scholar
• Wilson, M., & Gochyyev, P. (2020). Having your cake and eating it too: Multiple dimensions and a composite. Measurement, 151, 107247. https://doi.org/10.1016/j.measurement.2019.107247
   Web of Science ®Google Scholar
• Yang, Y., & Hu, B. Y. (2019). Chinese preschool teachers’ classroom instructional support quality and child-centered beliefs: A latent profile analysis. Teaching and Teacher Education, 80, 1–12. https://doi.org/10.1016/j.tate.2018.12.021
   Web of Science ®Google Scholar
• Yarkoni, T. (2020). Implicit realism impedes progress in psychology: Comment on fried (2020). Psychological Inquiry, 31(4), 326–333. https://doi.org/10.1080/1047840X.2020.1853478
   Web of Science ®Google Scholar