Estimating School Climate Traits Across Multiple Informants: An Illustration of a Multitrait–Multimethod Validation Through Latent Variable Modeling

References

• Achenbach, T. M., McConaughy, S. H., & Howell, C. T. (1987). Child/adolescent behavioral and emotional problems: Implications of cross informant correlations for situational specificity. Psychological Bulletin, 101, 213–232.
   PubMed Web of Science ®Google Scholar
• Aiken, L. H., & Patrician, P. A. (2000). Measuring organizational traits of hospitals: The revised nursing work index. Nursing Research, 49, 146–153.
   PubMed Web of Science ®Google Scholar
• Anderson, E. M. (2002). School effects on psychological outcomes during adolescence. Journal of Educational Psychology, 94, 795–805.
   Web of Science ®Google Scholar
• Austin, G., & Duerr, M. (2005). Guidebook for the California Healthy Kids Survey. Part III. School Climate Survey for teachers and other staff (2005–2006 ed.). San Francisco, CA: WestEd. Retrieved from http://www.wested.org/pub/docs/chks_surveys_summary.html
   Google Scholar
• Bandyopadhyay, S., Cornell, D., & Konold, T. (2009). Internal and external validity of three school climate scales from the School Climate Bullying Survey. School Psychology Review, 38, 338–355.
   Web of Science ®Google Scholar
• Bauer, D. J., Howard, A. L., Baldasaro, R. E., Curran, P. J., Hussong, A. M., & Chassin, L. (2013). A trifactor model for integrating ratings across multiple informants. Psychological Methods, 18, 475–493.
   PubMed Web of Science ®Google Scholar
• Bentler, P. M., & Bonett, D. G. (1980). Significance tests and goodness-of-fit in the analysis of covariance structures. Psychological Bulletin, 88, 588–606.
   Web of Science ®Google Scholar
• Bleidorn, W., & Peters, A. L. (2011). A multilevel multitrait-multimethod analysis of self- and peer-reported daily affective experiences. European Journal of Personality, 25, 398–408.
   Web of Science ®Google Scholar
• Browne, M., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. Bollen & J. Long (Eds.), Testing structural equation models (pp. 136–162). Newbury Park, CA: Sage.
   Google Scholar
• Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait–multimethod matrix. Psychological Bulletin, 56, 81–105.
   PubMed Web of Science ®Google Scholar
• Carretero-Dios, H., Eid, M., & Ruch, W. (2011). Analyzing multitrait-multimethod data with multilevel confirmatory factor analysis: An application to the validation of the State-Trait Cheerfulness inventory. Journal of Research in Personality, 45, 153–164.
   Web of Science ®Google Scholar
• Cornell, D., Gregory, A., Huang, F., & Fan, X. (2013). Perceived prevalence of bullying and teasing predicts high school dropout rates. Journal of Educational Psychology, 105, 138–149.
   Web of Science ®Google Scholar
• Cornell, D., Huang, F., Konold, T., Meyer, P., Shukla, K., Heilbrun, A., … Nekvasil, E. (2014). Technical Report of the Virginia Secondary School Climate Survey: 2014 Results for 9th–12th grade students and teachers. Charlottesville: Curry School of Education, University of Virginia.
   Google Scholar
• Cornell, D., Shukla, K., & Konold, T. (2015, April 27). Peer victimization and authoritative school climate: a multilevel approach. Journal of Educational Psychology. Advance online publication. doi:10.1037/edu0000038
   Google Scholar
• Cornell, D., Shukla, K., & Konold, T. (in press). Authoritative school climate and student academic engagement, grades, and aspirations in middle and high schools. AERA Open, 2, 1–18. doi:10.1177/2332858416633184
   Google Scholar
• De Los Reyes, A., Augenstein, T. M., Wang, M., Thomas, S. A., Drabick, D., Burgers, D. E., & Rabinowitz, J. (2015). The validity of the multi-informant approach to assessing child and adolescent mental health. Psychological Bulletin, 141, 858–900.
   PubMed Web of Science ®Google Scholar
• Dyer, N. G., Hanges, P. J., & Hall, R. J. (2005). Applying multilevel confirmatory factor analysis techniques to the study of leadership. Leadership Quarterly, 16, 149–167.
   Web of Science ®Google Scholar
• Eid, M., Nussbeck, F. W., Geiser, C., Cole, D. A., Gollwitzer, M., & Lischetzke, T. (2008). Structural equation modeling of multitrait-multimethod data: Different models for different types of methods. Psychological Methods, 13, 230–253.
   PubMed Web of Science ®Google Scholar
• Enticott, G., Boyne, G. A., & Walker, R. M. (2009). The use of multiple informants in public administration research: Data aggregation using organizational echelons. Journal of Public Administration Research and Theory, 19, 229–253.
   Web of Science ®Google Scholar
• Fan, X., Thompson, B., & Wang, L. (1999). Effects of sample size, estimation methods, and model specification on structural equation modeling fit indexes. Structural Equation Modeling, 6, 56–83.
   Web of Science ®Google Scholar
• Gregory, A., Cornell, D., Fan, X., Sheras, P., & Shih, T. (2010). Authoritative school discipline: High school practices associated with lower student bullying and victimization. Journal of Educational Psychology, 102, 483–496.
   Web of Science ®Google Scholar
• Guion, R. M., & Ironson, G. H. (1983). Latent trait theory for organizational research. Organizational Behavior and Human Performance, 31, 54–87.
   Web of Science ®Google Scholar
• Hu, L., & Bentler, P. M. (1995). Evaluating model fit. In R. H. Hoyle (Ed.), Structural equation modeling: Concepts, issues, and applications (pp. 76–99). Thousand Oaks, CA: Sage.
   Google Scholar
• Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55. doi:10.1080/10705519909540118
   Web of Science ®Google Scholar
• Huang, F., & Cornell. D. (2016). Multilevel factor structure, concurrent validity, and test–retest reliability of the high school teacher version of the Authoritative School Climate Survey. Journal of Psychoeducational Assessment, 34, 536–549. doi:10.1177/0734282915621439
   Web of Science ®Google Scholar
• Huang, F., Cornell, D., Konold, R., Meyer, P., Lacey, A., Nekvasil, E., … Shukla, K. (2015). Multilevel factor structure and concurrent validity of the teacher version of the Authoritative School Climate Survey. Journal of School Health, 85, 843–851.
   PubMed Web of Science ®Google Scholar
• Juvonen, J., Nishina, A., & Graham, S. (2000). Peer harassment, psychological adjustment, and school functioning in early adolescence. Journal of Educational Psychology, 92, 349–359.
   Web of Science ®Google Scholar
• Kearney, W. S., & Peters, S. (2013). A comparison of teacher and student perceptions of elementary classroom climate. National Forum of Educational Administration and Supervision, 31, 20–37.
   Google Scholar
• Klein, J., Cornell, D., & Konold, T. (2013). Relationships between bullying, school climate, and student risk behaviors. School Psychology Quarterly, 27, 154–169.
   Web of Science ®Google Scholar
• Koch, T., Schultze, M., Burrus, J., Roberts, R. D., & Eid, M. (2015). A multilevel CFA-MTMM models for nested structurally different methods. Journal of Educational and Behavioral Statistics, 40, 477–510.
   Web of Science ®Google Scholar
• Konold, T. R., & Cornell, D. (2015a). Measurement and structural relations of an authoritative school climate model: A multi-level latent variable investigation. Journal of School Psychology. Advance online publication. doi:10.1016/j.jsp.2015.09.001
   Google Scholar
• Konold, T. R., & Cornell, D. (2015b). Multilevel multitrait–multimethod model of structurally different and interchangeable raters of school climate. Psychological Assessment. Advance online publication. doi:10.1037/pas0000098
   Google Scholar
• Konold, T. R., Cornell, D., Huang, F., Meyer, P., Lacey, A., Nekvasil, E., … Shukla, K. (2014). Multi-level multi-informant structure of the Authoritative School Climate Survey. School Psychology Quarterly, 29, 238–255.
   Google Scholar
• Konold, T. R., & Glutting, J. J. (2008). ADHD and method variance: A latent variable approach applied to a nationally representative sample of college freshman. Journal of Learning Disabilities, 41, 405–416.
   PubMed Web of Science ®Google Scholar
• Konold, T. R., & Pianta, R. C. (2007). The influence of informants’ on ratings of children’s behavioral functioning: A latent variable approach. Journal of Psychoeducational Assessment, 25, 222–236.
   Web of Science ®Google Scholar
• Konold, T. R., & Shukla, K. D. (2014). Informant effects on behavioral and academic associations: A latent variable longitudinal examination. Psychology in the Schools, 51, 554–566.
   Web of Science ®Google Scholar
• Kooij, J. S., Boonstra, A. M., Swinkels, S. H. N., Bekker, E. M., de Noord, I., & Buitelaar, J. K. (2008). Reliability, validity, and utility of instruments for self-report and informant report concerning symptoms of ADHD in adult patients. Journal of Attention Disorders, 11, 445–458.
   Google Scholar
• Kumar, N., Stern, L. W., & Anderson, J. C. (1993). Conducting interorganizational research using key informants. Academy of Management Journal, 36, 1633–1651.
   Web of Science ®Google Scholar
• Lance, C. E., Noble, C. L., & Scullen, S. E. (2002). A critique of the correlated trait-correlated method and correlated uniqueness models for multitrait-multimethod data. Psychological Methods, 7, 228–244.
   PubMed Web of Science ®Google Scholar
• Luo, Z., Breslau, J., Gardiner, J. C., Chen, Q., & Breslau, N. (2014). Assessing interchangeability at cluster levels with multiple-informant data. Statistics in Medicine, 33, 361–375. doi:10.1002/sim.5948.
   PubMed Web of Science ®Google Scholar
• Marsh, H. W., Ludtke, O., Nagengast, B., Trautwein, U, Morin, A. J. S., Abduljabbar, A. S., & Koller, O. (2012). Classroom climate and contextual effects: Conceptual and methodological issues in the evaluation of group-level effects. Educational Psychologist, 47, 106–124.
   Web of Science ®Google Scholar
• Mehta, S., Cornell, D., Fan, X., & Gregory, A. (2013). Bullying climate and school engagement in ninth grade students. Journal of School Health, 83, 45–54.
   PubMed Web of Science ®Google Scholar
• Nussbeck, F. W., Eid, M., Geiser, C., Courvoisier, D. S., & Lischetzke, T. (2009). A CTC(M-1) model for different types of raters, Methodology, 5, 88–98.
   Web of Science ®Google Scholar
• Olweus, D. (2007). The Olweus Bullying Questionnaire. Center City, MN: Hazelden.
   Google Scholar
• Pellerin, L. A. (2005). Student disengagement and the socialization styles of high schools. Social Forces, 84, 1161–1179.
   Web of Science ®Google Scholar
• Rajmil, L., Lopez, A. R., Lopez-Aquila, S., & Alonso, J. (2013). Parent–child agreement on health-related quality of life (HRQOL): A longitudinal study. Health and Quality of Life Outcomes, 11, 101.
   PubMed Web of Science ®Google Scholar
• Shukla, K., Konold, T., & Cornell, D. (2016). Profiles of student perceptions of school climate: relations with risk behaviors and academic outcomes. American Journal of Community Psychology, 57, 291–307. doi:10.1002/ajcp.12044.
   Google Scholar
• Shukla, K. D., & Wiesner, M. (2015). Direct and indirect violence exposure: relations to depression for economically disadvantaged ethnic minority mid-adolescents. Violence and Victims, 30, 120–135.
   PubMed Web of Science ®Google Scholar
• Thornberry, T. P., Lizotte, A. J., Krohn, M. D., Farnworth, M., & Jang, S. J. (1991). Testing interactional theory: An examination of reciprocal causal relationships among family, school, and delinquency. Journal of Criminal Law & Criminology, 82, 3–35.
   Web of Science ®Google Scholar
• Van Bruggen, G. H., Lilien, G. L., & Kacker, M. (2002). Informants in organizational marketing research: Why use multiple informants and how to aggregate responses. Journal of Marketing Research, 39, 469–478.
   Web of Science ®Google Scholar
• Vaughn, A. J., & Hoza, B. (2013). The incremental utility of behavioral rating scales and a structured diagnostic interview in the assessment of attention-deficit/hyperactivity disorder. Journal of Emotional and Behavioral Disorders, 21, 227–239. doi: 10.1177/1063426611427910
   Web of Science ®Google Scholar
• Vazire, S. (2006). Informant reports: A cheap, fast, and easy method for personality assessment. Journal of Research in Personality, 40, 472–481.
   Web of Science ®Google Scholar
• Virginia Department of Education. (n.d.). Retrieved from http://www.doe.virginia.gov/directories/sch_division_locales_schedules/school_division_locale_descriptions.pdf
   Google Scholar
• Wang, M. T., & Degol, J. L. (2015). School climate: A review of the construct, measurement, and impact on student outcomes. Educational Psychology Review. Advance online publication. doi: 10.1007/s10648-015-9319-1
   Google Scholar