Exploring the reliability of generic and content-specific instructional aspects in physical education lessons

References

• Ashy, M. H., Lee, A. M., & Landin, D. K. (1988). Relationship of practice using correct technique to achievement in a motor skill. Journal of Teaching in Physical Education, 7, 115–120. doi:10.1123/jtpe.7.2.115
   Google Scholar
• Blömeke, S., Busse, A., Kaiser, G., König, J., & Suhl, U. (2016). The relation between content-specific and general teacher knowledge and skills. Teaching and Teacher Education, 56, 35–46. doi:10.1016/j.tate.2016.02.003
   Web of Science ®Google Scholar
• Bottema-Beutel, K., Lloyd, B., Carter, E. W., & Asmus, J. M. (2014). Generalizability and decision studies to inform observational and experimental research in classroom settings. American Journal of Intellectual and Developmental Disabilities, 119, 589–605. doi:10.1352/1944-7558-119.6.589
   PubMedGoogle Scholar
• Brennan, R. L. (2001). Generalizability theory. New York, NY: Springer.
   Google Scholar
• Briesch, A. M., Swaminathan, H., Welsh, M., & Chafouleas, S. M. (2014). Generalizability theory: A practical guide to study design, implementation, and interpretation. Journal of School Psychology, 52, 13–35. doi:10.1016/j.jsp.2013.11.008
   PubMed Web of Science ®Google Scholar
• Buck, M., Harrison, J. M., & Bryce, G. R. (1991). An analysis of learning trials and their relationship to achievement in volleyball. Journal of Teaching in Physical Education, 10, 134–152. doi:10.1123/jtpe.10.2.134
   Web of Science ®Google Scholar
• Cantrell, S., & Kane, T. J. (2013). Ensuring fair and reliable measures of effective teaching: Culminating findings from the MET project’s three-year study. Seattle, WA: Bill & Melinda Gates Foundation.
   Google Scholar
• Casabianca, J. M., Lockwood, J. R., & McCaffrey, D. F. (2015). Trends in classroom observation scores. Educational and Psychological Measurement, 75, 311–337. doi:10.1177/0013164414539163
   PubMed Web of Science ®Google Scholar
• Casabianca, J. M., McCaffrey, D. F., Gitomer, D. H., Bell, C. A., Hamre, B. K., & Pianta, R. C. (2013). Effect of observation mode on measures of secondary mathematics teaching. Educational and Psychological Measurement, 73, 757–783. doi:10.1177/0013164413486987
   Web of Science ®Google Scholar
• Charalambous, C. Y., & Kyriakides, E. (2017). Working at the nexus of generic and content-specific teaching practices: An exploratory study based on TIMSS secondary analyses. The Elementary School Journal, 117, 423–454. doi:10.1086/690221
   Web of Science ®Google Scholar
• Chen, W., Hendricks, K., & Archibald, K. (2011). Assessing preservice teachers’ quality teaching practices. Educational Research and Evaluation, 17, 13–32. doi:10.1080/13803611.2011.578008
   Google Scholar
• Chen, W., Mason, S., Staniszewski, C., Upton, A., & Valley, M. (2012). Assessing the quality of teachers’ teaching practices. Educational Assessment, Evaluation and Accountability, 24, 25–41. doi:10.1007/s11092-011-9134-2
   Web of Science ®Google Scholar
• Chiu, C. W. T., & Wolfe, E. W. (2002). A method for analyzing sparse data matrices in the generalizability theory framework. Applied Psychological Measurement, 26, 321–338. doi:10.1177/0146621602026003006
   Web of Science ®Google Scholar
• Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6, 284–290. doi:10.1037/1040-3590.6.4.284
   Google Scholar
• Constantinides, P., Montalvo, R., & Silverman, S. (2013). Teaching processes in elementary physical education classes taught by specialists and nonspecialists. Teaching and Teacher Education, 36, 68–76. doi:10.1016/j.tate.2013.07.004
   Web of Science ®Google Scholar
• Cor, M. K. (2011). Investigating the reliability of classroom observation protocols: The case of PLATO. Retrieved from http://platorubric.stanford.edu/Cor%20M%20K%20%20(2011).pdf
   Google Scholar
• Creemers, B. P. M., & Kyriakides, L. (2008). The dynamics of educational effectiveness: A contribution to policy, practice and theory in contemporary schools. London, UK: Routledge.
   Google Scholar
• Danielson, C. (2013). Enhancing professional practice: A framework for teaching (2nd ed.). Alexandria, VA: Association for Supervision and Curriculum Development.
   Google Scholar
• Douglas, K. (2009). Sharpening our focus in measuring classroom instruction. Educational Researcher, 38, 518–521. doi:10.3102/0013189X09350881
   Web of Science ®Google Scholar
• Goodwin, L. D. (2001). Interrater agreement and reliability. Measurement in Physical Education and Exercise Science, 5, 13–34. doi:10.1207/S15327841MPEE0501_2
   Google Scholar
• Grossman, P., Loeb, S., Cohen, J., Hammerness, K., Wyckoff, J., Boyd, D., & Lankford, H. (2010). Measure for measure: The relationship between measures of instructional practice in middle school English language arts and teachers’ value-added scores (NBER Working Paper No. 16015). Cambridge, MA: National Bureau of Economic Research.
   Google Scholar
• Herlihy, C., Karger, E., Pollard, C., Hill, H., Kraft, M., Williams, M., & Howard, S. (2014). State and local efforts to investigate the validity and reliability of scores from teacher evaluation systems. Teachers College Record, 116, 2–28.
   Web of Science ®Google Scholar
• Hill, H. C., Blunk, M. L., Charalambous, C. Y., Lewis, J. M., Phelps, G. C., Sleep, L., & Ball, D. L. (2008). Mathematical knowledge for teaching and the mathematical quality of instruction: An exploratory study. Cognition and Instruction, 26, 430–511. doi:10.1080/07370000802177235
   Web of Science ®Google Scholar
• Hill, H. C., Charalambous, C. Y., & Kraft, M. A. (2012). When rater reliability is not enough: Teacher observation systems and a case for the generalizability study. Educational Researcher, 41, 56–64. doi:10.3102/0013189X12437203
   Web of Science ®Google Scholar
• Ho, A. D., & Kane, T. J. (2013). The reliability of classroom observations by school personnel. Seattle, WA: Bill & Melinda Gates Foundation.
   Google Scholar
• Hunt, G. H., Wiseman, D. G., & Touzel, T. J. (2009). Effective teaching: Preparation and implementation (4th ed.). Springfield, IL: Charles C. Thomas.
   Google Scholar
• Kane, T. J., & Staiger, D. O. (2012). Gathering feedback for teaching: Combining high-quality observations with student surveys and achievement gains. Seattle, WA: Bill & Melinda Gates Foundation.
   Google Scholar
• Kennedy, M. M. (2010). Approaches to annual performance assessment. In M. M. Kennedy (Ed.), Teacher assessment and the quest for teacher quality: A handbook (pp. 225–250). San Francisco, CA: Jossey-Bass.
   Google Scholar
• Klieme, E., Pauli, C., & Reusser, K. (2009). The Pythagoras study: Investigating effects of teaching and learning in Swiss and German mathematics classrooms. In T. Janík & T. Seidel (Eds.), The power of video studies in investigating teaching and learning in the classroom (pp. 137–160). Münster, Germany: Waxmann.
   Google Scholar
• Konstantopoulos, S. (2012). Teacher effects: Past, present and future. In S. Kelly (Ed.), Assessing teacher quality: Understanding teacher effects on instruction and achievement (pp. 33–48). New York, NY: Teachers College Press.
   Google Scholar
• Kyriakides, E. (2016). Combining generic and content-specific practices in exploring teaching quality in Physical Education and its impact on student learning ( Unpublished doctoral dissertation). University of Cyprus, Nicosia, Cyprus.
   Google Scholar
• Kyriakides, L., & Tsangaridou, N. (2008). Towards the development of generic and differentiated models of educational effectiveness: A study on school and teacher effectiveness in physical education. British Educational Research Journal, 34, 807–838. doi:10.1080/01411920802041467
   Web of Science ®Google Scholar
• Lindsay, E. L. (2014). Effective teaching in physical education: The view from a variety of trenches. Research Quarterly for Exercise and Sport, 85, 31–37. doi:10.1080/02701367.2014.873330
   PubMed Web of Science ®Google Scholar
• Magill, R. A. (2010). Motor learning and control: Concepts and applications (9th ed.). New York, NY: McGraw-Hill.
   Google Scholar
• Martinéz, J. F., Borko, H., & Stecher, B. M. (2012). Measuring instructional practice in science using classroom artifacts: Lessons learned from two validation studies. Journal of Research in Science Teaching, 49, 38–67. doi:10.1002/tea.20447
   Web of Science ®Google Scholar
• Mashburn, A. J., Meyer, J. P., Allen, J. P., & Pianta, R. C. (2014). The effect of observation length and presentation order on the reliability and validity of an observational measure of teaching quality. Educational and Psychological Measurement, 74, 400–422. doi:10.1177/0013164413515882
   Web of Science ®Google Scholar
• Masser, L. S. (1993). Critical cues help first-grade students’ achievement in handstands and forward rolls. Journal of Teaching in Physical Education, 12, 301–312. doi:10.1123/jtpe.12.3.301
   Web of Science ®Google Scholar
• McKenzie, T. L., & van der Mars, H. (2015). Top 10 research questions related to assessing activity and its context using systematic observation. Research Quarterly for Exercise and Sport, 86, 13–29. doi:10.1080/02701367.2015.991264
   PubMed Web of Science ®Google Scholar
• Meyer, J. P., Liu, X., & Mashburn, A. J. (2014). A practical solution to optimizing the reliability of teaching observation measures under budget constraints. Educational and Psychological Measurement, 74, 280–291. doi:10.1177/0013164413508774
   Web of Science ®Google Scholar
• Muijs, D., Kyriakides, L., Van der Werf, G., Creemers, B., Timperley, H., & Earl, L. (2014). State of the art – Teacher effectiveness and professional learning. School Effectiveness and School Improvement, 25, 231–256. doi:10.1080/09243453.2014.885451
   Web of Science ®Google Scholar
• Muijs, D., & Reynolds, D. (2003). Student background and teacher effects on achievement and attainment in mathematics: A longitudinal study. Educational Research and Evaluation, 9, 289–314. doi:10.1076/edre.9.3.289.15571
   Google Scholar
• Newton, X. A. (2010). Developing indicators of classroom practice to evaluate the impact of district mathematics reform initiative: A generalizability analysis. Studies in Educational Evaluation, 36, 1–13. doi:10.1016/j.stueduc.2010.10.002
   Google Scholar
• O’Sullivan, M., & Dyson, B. (1994). Rules, routines, and expectations of 11 high school physical education teachers. Journal of Teaching in Physical Education, 13, 361–374. doi:10.1123/jtpe.13.4.361
   Web of Science ®Google Scholar
• Papay, J. P. (2012). Refocusing the debate: Assessing the purposes and tools of teacher evaluation. Harvard Educational Review, 82, 123–141. doi:10.17763/haer.82.1.v40p0833345w6384
   Web of Science ®Google Scholar
• Paye, B., & Paye, P. (2013). Youth basketball drills (2nd ed.). Champaign, IL: Human Kinetics.
   Google Scholar
• Pianta, R., & Hamre, B. K. (2009). Conceptualization, measurement, and improvement of classroom processes: Standardized observation can leverage capacity. Educational Researcher, 38, 109–119. doi:10.3102/0013189X09332374
   Web of Science ®Google Scholar
• Praetorius, A.-K., Pauli, C., Reusser, K., Rakoczy, K., & Klieme, E. (2014). One lesson is all you need? Stability of instructional quality across lessons. Learning and Instruction, 31, 2–12. doi:10.1016/j.learninstruc.2013.12.002
   Web of Science ®Google Scholar
• Reynolds, D., Chapman, C., Clarke, P., Muijs, D., Sammons, P., & Teddlie, C. (2016). Conclusions: The future of educational effectiveness and improvement research, and some suggestions and speculations. In C. Chapman, D. Muijs, D. Reynolds, P. Sammons, & C. Teddlie (Eds.), The Routledge international handbook of educational effectiveness and improvement: Research, policy, and practice (pp. 408–439). New York, NY: Routledge.
   Google Scholar
• Rink, J. E. (2003). Effective instruction in physical education. In S. J. Silverman & C. D. Ennis (Eds.), Student learning in physical education: Applying research to enhance instruction (2nd ed., pp. 165–186). Champaign, IL: Human Kinetics.
   Google Scholar
• Rink, J. E. (2010). Teaching physical education for learning (6th ed.). New York, NY: McGraw-Hill.
   Google Scholar
• Rink, J. E. (2013). Measuring teacher effectiveness in physical education. Research Quarterly for Exercise and Sport, 84, 407–418. doi:10.1080/02701367.2013.844018
   PubMed Web of Science ®Google Scholar
• Rink, J. E., & Werner, P. (1989). Qualitative measures of teaching performance scale. In P. W. Darst, D. Zakrajsek, & V. H. Mancini (Eds.), Analyzing physical education and sport instruction (2nd ed., pp. 269–276). Champaign, IL: Human Kinetics.
   Google Scholar
• Rowan, B., & Correnti, R. (2009). Studying reading instruction with teacher logs: Lessons from the study of instructional improvement. Educational Researcher, 38, 120–131. doi:10.3102/0013189X09332375
   Web of Science ®Google Scholar
• Sariscsany, M. J., Darst, P. W., & van der Mars, H. (1995). The effects of three teacher supervision patterns on student on-task and skill performance in secondary physical education. Journal of Teaching in Physical Education, 14, 179–197. doi:10.1123/jtpe.14.2.179
   Web of Science ®Google Scholar
• Schuldheisz, J. M., & van der Mars, H. (2002). Active supervision and students’ physical activity in middle school physical education. Journal of Teaching in Physical Education, 21, 75–90. doi:10.1123/jtpe.21.1.75
   Web of Science ®Google Scholar
• Seidel, T., & Shavelson, R. J. (2007). Teaching effectiveness research in the past decade: The role of theory and research design in disentangling meta-analysis results. Review of Educational Research, 77, 454–499. doi:10.3102/0034654307310317
   Web of Science ®Google Scholar
• Shavelson, R., & Dempsey-Atwood, N. (1976). Generalizability of measures of teaching behavior. Review of Educational Research, 46, 553–611. doi:10.3102/00346543046004553
   Web of Science ®Google Scholar
• Shavelson, R. J., & Webb, N. M. (1991). Generalizability theory: A primer. Thousand Oaks, CA: Sage.
   Google Scholar
• Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15(2), 4–14. doi:10.3102/0013189X015002004
   Google Scholar
• Siedentop, D., & Tannehill, D. (2000). Developing teaching skills in physical education (4th ed.). Mountain View, CA: Mayfield.
   Google Scholar
• Silverman, S. (1993). Student characteristics, practice, and achievement in physical education. The Journal of Educational Research, 87, 54–61. doi:10.1080/00220671.1993.9941166
   Web of Science ®Google Scholar
• Silverman, S. (2011). Teaching for student learning in physical education. Journal of Physical Education, Recreation & Dance, 82(6), 29–34. doi:10.1080/07303084.2011.10598642
   Google Scholar
• Silverman, S., Tyson, L., & Krampitz, J. (1992). Teacher feedback and achievement in physical education: Interaction with student practice. Teaching and Teacher Education, 8, 333–344. doi:10.1016/0742-051X(92)90060-G
   Web of Science ®Google Scholar
• Smit, N., Van de Grift, W., De Bot, K., & Jansen, E. (2017). A classroom observation tool for scaffolding reading comprehension. System, 65, 117–129. doi:10.1016/j.system.2016.12.014
   Web of Science ®Google Scholar
• Strong, M. (2011). The highly qualified teacher: What is teacher quality and how do we measure it? New York, NY: Teachers College Press.
   Google Scholar
• Stronge, J. H., Ward, T. J., & Grant, L. W. (2011). What makes good teachers good? A cross-case analysis of the connection between teacher effectiveness and student achievement. Journal of Teacher Education, 62, 339–355. doi:10.1177/0022487111404241
   Web of Science ®Google Scholar
• Tsangaridou, N. (2006). Teachers’ knowledge. In D. Kirk, D. Macdonald, & M. O’Sullivan (Eds.), The handbook of physical education (pp. 502–515). London, UK: Sage.
   Google Scholar
• Van de Grift, W. J. C. M. (2014). Measuring teaching quality in several European countries. School Effectiveness and School Improvement, 25, 295–311. doi:10.1080/09243453.2013.794845
   Web of Science ®Google Scholar
• Van der Lans, R. M., Van de Grift, W. J. C. M., Van Veen, K., & Fokkens-Bruinsma, M. (2016). Once is not enough: Establishing reliability criteria for feedback and evaluation decisions based on classroom observations. Studies in Educational Evaluation, 50, 88–95. doi:10.1016/j.stueduc.2016.08.001
   Web of Science ®Google Scholar
• van der Mars, H. (2006). Time and learning in physical education. In D. Kirk, D. Macdonald, & M. O’Sullivan (Eds.), The handbook of physical education (pp. 191–213). London, UK: Sage.
   Google Scholar
• Walkowiak, T. A., Berry, R. Q., Meyer, J. P., Rimm-Kaufman, S. E., & Ottmar, E. R. (2014). Introducing an observational measure of standards-based mathematics teaching practices: Evidence of validity and score reliability. Educational Studies in Mathematics, 85, 109–128. doi:10.1007/s10649-013-9499-x
   Web of Science ®Google Scholar
• Ward, P. (2009). Content matters: Knowledge that alters teaching. In L. D. Housner, M. W. Metzler, P. G. Schempp, & T. J. Templin (Eds.), Historic traditions and future directions of research on teaching and teacher education in physical education (pp. 345–356). Morgantown, WV: Fitness Information Technology.
   Google Scholar
• Ward, P., Kim, I., Ko, B., & Li, W. (2015). Effects of improving teachers’ content knowledge on teaching and student learning in physical education. Research Quarterly for Exercise and Sport, 86, 130–139. doi:10.1080/02701367.2014.987908
   PubMed Web of Science ®Google Scholar
• Wilhelm, A. G., & Kim, S. (2015). Generalizing from observations of mathematics teachers’ instructional practice using the instructional quality assessment. Journal for Research in Mathematics Education, 46, 270–279. doi:10.5951/jresematheduc.46.3.0270
   Web of Science ®Google Scholar
• Wood, J., Tocci, C. M., Joe, J. N., Holtzman, S. L., Cantrell, S., & Archer, J. (2014). Building trust in observations: A blueprint for improving systems to support great teaching. Princeton, NJ: Bill & Melinda Gates Foundation.
   Google Scholar
• Yiallourides, G. (1998). Factors influencing the attitudes of 9 to 14 year old Cypriot pupils towards Physical Education (Unpublished PhD Thesis). University of Manchester, Manchester, UK.
   Google Scholar