Development of scientific reasoning test measuring control of variables strategy in physics for high school students: evidence of validity and latent predictors of item difficulty

References

• Adams, R., & August, M. W. (2010). Modelling a dichotomously scored multiple choice test with the Rasch model (Issue August). ConQuest.
   Google Scholar
• Adey, P., & Csapó, B. (2012). Developing and assessing scientific reasoning. In B. Csapó & G. Szabó (Eds.), Framework for diagnostic assessment of science (pp. 17–53). Nemzeti Tankönyvkiadó.
   Google Scholar
• American Association for the Advancement of Science (AAAS). (2012). AAAS Science Assessment - Project2061. https://www.aaas.org/programs/project-2061
   Google Scholar
• American Education Research Association, American Psychological Association, and N. M. in E. (1999). Standards for educational and psychological testing. American Psychological Association. https://www.aera.net/Portals/38/1999Standards_revised.pdf
   Google Scholar
• Bao, L., Xiao, Y., Koenig, K., & Han, J. (2018). Validity evaluation of the Lawson classroom test of scientific reasoning. Physical Review Physics Education Research, 14(2), 1–19. https://doi.org/10.1103/PhysRevPhysEducRes.14.020106
   Web of Science ®Google Scholar
• Boudreaux, A., Shaffer, P. S., Heron, P. R. L., & McDermott, L. C. (2008). Student understanding of control of variables: Deciding whether or not a variable influences the behavior of a system. American Journal of Physics, 76(2), 163–170. https://doi.org/10.1119/1.2805235
   Web of Science ®Google Scholar
• Boujaoude, S., Salloum, S., & Abd-El-Khalick, F. (2004). Relationships between selective cognitive variables and students’ ability to solve chemistry problems. International Journal of Science Education, 26(1), 63–84. https://doi.org/10.1080/0950069032000070315
   Web of Science ®Google Scholar
• Chen, Z., & Klahr, D. (1999). All other things being equal: Acquisition and transfer of the control of variables strategy. Child Development, 70(5), 1098–1120. https://doi.org/10.1111/1467-8624.00081
   PubMed Web of Science ®Google Scholar
• Csapó, B., & Molnár, G. (2019). Online diagnostic assessment in support of personalized teaching and learning: The eDia system. Frontiers in Psychology, 10, 1522. https://doi.org/10.3389/fpsyg.2019.01522
   PubMed Web of Science ®Google Scholar
• Ding, L., Wei, X., & Mollohan, K. (2016). Does higher education improve student scientific reasoning skills? International Journal of Science and Mathematics Education, 14(4), 619–634. https://doi.org/10.1007/s10763-014-9597-y
   Web of Science ®Google Scholar
• Dunn, T. J., Baguley, T., & Brunsden, V. (2014). From alpha to omega: A practical solution to the pervasive problem of internal consistency estimation. British Journal of Psychology, 105(3), 399–412. https://doi.org/10.1111/bjop.12046
   PubMed Web of Science ®Google Scholar
• Ebel, R. L., & Frisbie, D. A. (1991). Essentials of educational measurement (Vol. 11, Issue 2). Prentice-Hall.
   Google Scholar
• Edelsbrunner, P. A., & Dablander, F. (2019). The psychometric modeling of scientific reasoning: A review and recommendations for future avenues. Educational Psychology Review, 31(1), 1–34. http://doi.org/10.1007/s10648-018-9455-5
   Web of Science ®Google Scholar
• Edelsbrunner, P. A., Schalk, L., Schumacher, R., & Stern, E. (2018). Variable control and conceptual change: A large-scale quantitative study in elementary school. Learning and Individual Differences, 66, 38–53. https://doi.org/10.1016/j.lindif.2018.02.003
   Web of Science ®Google Scholar
• Griffin, P. (2010). Item response modelling: An introduction to the Rasch model. Assessment Research Centre Faculty of Education, The University of Melbourne.
   Google Scholar
• Han, J. (2013). Scientific reasoning: Research, development, and assessment [Unpublished doctoral dissertation]. The Ohio State University.
   Google Scholar
• Hejnová, E., Eisenmann, P., Cihlář, J., & Přibyl, J. (2018). Relations between scientific reasoning and culture of problem solving. Journal on Efficiency and Responsibility in Education and Science, 11(2), 38–44. https://doi.org/10.7160/eriesj.2018.110203
   Web of Science ®Google Scholar
• Kane, M. T. (2016). Explicating validity. Assessment in Education: Principles, Policy & Practice, 23(2), 198–211. https://doi.org/10.1080/0969594X.2015.1060192
   Web of Science ®Google Scholar
• Kaygısız, G. M., & Gürkan, B. (2018). Adaptation of scientific reasoning scale into turkish and examination of its psychometric properties. Educational Sciences: Theory & Practice, 18(3), 737–757. https://doi.org/10.12738/estp.2018.3.0175
   Google Scholar
• Kuhn, D. (2007). Reasoning about multiple variables: Control of variables is not the only challenge. Science Education, 91(5), 710–726. https://doi.org/10.1002/sce.20214
   Web of Science ®Google Scholar
• Lawson, A. (2000). Classroom test of scientific reasoning. In Revised Edition: August 2000 by Anton E. Lawson, Arizona State University. Based on: Lawson, A.E. 1978. Development and validation of the classroom test of formal reasoning. Journal of Research in Science Teaching, 15(1): 11-24. http://www.public.asu.edu/~anton1/AssessArticles/Assessments/MathematicsAssessments/Scientific Reasoning Test.pdf
   Google Scholar
• Luecht, R. M., & Sireci, S. G. (2011). A review of models for computer-based testing. Research Report 2011-12. College Board. www.collegeboard.org
   Google Scholar
• Magis, D., Béland, S., Tuerlinckx, F., & De Boeck, P. (2010). A general framework and an R package for the detection of dichotomous differential item functioning. Behavior Research Methods, 42(3), 847–862. https://doi.org/10.3758/BRM.42.3.847
   PubMed Web of Science ®Google Scholar
• Mayer, D., Sodian, B., Koerber, S., & Schwippert, K. (2014). Scientific reasoning in elementary school children: Assessment and relations with cognitive abilities. Learning and Instruction, 29, 43–55. https://doi.org/10.1016/j.learninstruc.2013.07.005
   Web of Science ®Google Scholar
• Messick, S. (1995). Standards of validity and the validity of standards in performance asessment. Educational Measurement: Issues and Practice, 14(4), 5–8. https://doi.org/10.1111/j.1745-3992.1995.tb00881.x
   Google Scholar
• MOET. (2009). Tài liệu phân phối chương trình Vật lí THPT.
   Google Scholar
• Opitz, A., Heene, M., & Fischer, F. (2017). Measuring scientific reasoning – A review of test instruments. Educational Research and Evaluation, 23(3–4), 78–101. https://doi.org/10.1080/13803611.2017.1338586
   Web of Science ®Google Scholar
• Osborne, J. (2013). The 21st century challenge for science education: Assessing scientific reasoning. Thinking Skills and Creativity, 10, 265–279. https://doi.org/10.1016/j.tsc.2013.07.006
   Web of Science ®Google Scholar
• Piraksa, C., Srisawasdi, N., & Koul, R. (2014). Effect of gender on student’s scientific reasoning ability: A case study in Thailand. Procedia - Social and Behavioral Sciences, 116, 486–491. https://doi.org/10.1016/j.sbspro.2014.01.245
   Google Scholar
• Revelle, W. (2019). psych: Procedures for psychological, psychometric, and personality research. Northwestern University. https://cran.r-project.org/package=psych
   Google Scholar
• Schauble, L., Klopfer, L. E., & Raghavan, K. (1991). Students’ transition from an engineering model to a science model of experimentation. Journal of Research in Science Teaching, 28(9), 859–882. https://doi.org/10.1002/tea.3660280910
   Web of Science ®Google Scholar
• Schwichow, M., Christoph, S., Boone, W. J., & Härtig, H. (2016). The impact of sub-skills and item content on students’ skills with regard to the control-of-variables strategy. International Journal of Science Education, 38(2), 216–237. https://doi.org/10.1080/09500693.2015.1137651
   Web of Science ®Google Scholar
• Schwichow, M., Croker, S., Zimmerman, C., Höffler, T., & Härtig, H. (2016). Teaching the control-of-variables strategy: A meta-analysis. Developmental Review, 39, 37–63. https://doi.org/10.1016/j.dr.2015.12.001
   Web of Science ®Google Scholar
• Schwichow, M., Osterhaus, C., & Edelsbrunner, P. A. (2020). The relation between the control-of-variables strategy and content knowledge in physics in secondary school. Contemporary Educational Psychology, 63, 101923. https://doi.org/10.1016/j.cedpsych.2020.101923
   Web of Science ®Google Scholar
• Siler, S. A., & Klahr, D. (2012). Detecting, classifying, and remediating: Children’s explicit and implicit misconceptions about experimental design. In R. W. Proctor & E. J. Capaldi (Eds.), Psychology of science (pp. 137–180). Oxford University Press. https://doi.org/10.1093/acprof:oso/9780199753628.003.0007
   Google Scholar
• Song, J., & Black, P. J. (1992). The effects of concept requirements and task contexts on pupils’ performance in control of variables. International Journal of Science Education, 14(1), 83–93. https://doi.org/10.1080/0950069920140108
   Web of Science ®Google Scholar
• Staver, J. R. (1986). The effects of problem format, number of independent variables, and their interaction on student performance on a control of variables reasoning problem. Journal of Research in Science Teaching, 23(6), 533–542. https://doi.org/10.1002/tea.3660230606
   Web of Science ®Google Scholar
• Stender, A., Schwichow, M., Zimmerman, C., & Härtig, H. (2018). Making inquiry-based science learning visible: The influence of CVS and cognitive skills on content knowledge learning in guided inquiry. International Journal of Science Education, 40(15), 1812–1831. https://doi.org/10.1080/09500693.2018.1504346
   Web of Science ®Google Scholar
• Stevenson, C. E., Hickendorff, M., Resing, W. C. M., Heiser, W. J., & de Boeck, P. A. L. (2013). Explanatory item response modeling of children’s change on a dynamic test of analogical reasoning. Intelligence, 41(3), 157–168. https://doi.org/10.1016/j.intell.2013.01.003
   Web of Science ®Google Scholar
• Stiller, J., Hartmann, S., Mathesius, S., Straube, P., Tiemann, R., Nordmeier, V., Krüger, D., & Upmeier zu Belzen, A. (2016). Assessing scientific reasoning: A comprehensive evaluation of item features that affect item difficulty. Assessment and Evaluation in Higher Education, 41(5), 721–732. https://doi.org/10.1080/02602938.2016.1164830
   Web of Science ®Google Scholar
• Taber, K. S. (2018). The use of Cronbach’s alpha when developing and reporting research instruments in science education. Research in Science Education, 48(6), 1273–1296. https://doi.org/10.1007/s11165-016-9602-2
   Web of Science ®Google Scholar
• Tairab, H. H. (2015). Assessing students’ understanding of control of variables across three grade levels and gender. International Education Studies, 9(1), 44–54. https://doi.org/10.5539/ies.v9n1p44
   Google Scholar
• Tekkaya, C., & Yenilmez, A. (2006). Relationships among measures of learning orientation, reasoning ability, and conceptual understanding of photosynthesis and respiration in plants for grade 8 males and females. Journal of Elementary Science Education, 18(1), 1–14. https://doi.org/10.1007/BF03170650
   Google Scholar
• Thuneberg, H., Hautamäki, J., & Hotulainen, R. (2015). Scientific reasoning, school achievement and gender: A multilevel study of between and within school effects in Finland. Scandinavian Journal of Educational Research, 59(3), 337–356. https://doi.org/10.1080/00313831.2014.904426
   Web of Science ®Google Scholar
• TIMSS. (1997). TIMSS science items : Released set for population 2 (seventh and eighth grades). IEA TIMSS.
   Google Scholar
• Tobin, K. G., & Capie, W. (1981). The development and validation of a group test of logical thinking. Educational and Psychological Measurement, 41(2), 413–423. https://doi.org/10.1177/001316448104100220
   Web of Science ®Google Scholar
• Valanides, N. (1997). Cognitive abilities among twelfth-grade students: Implications for science teaching. Educational Research and Evaluation, 3(2), 160–186. https://doi.org/10.1080/1380361970030204
   Google Scholar
• Van Vo, D., & Csapó, B. (2020). Development of inductive reasoning in students across school grade levels. Thinking Skills and Creativity, 37(2020), 100699. https://doi.org/10.1016/j.tsc.2020.100699
   Google Scholar
• Wickham, H. (2016). Ggplot2: Elegant graphics for data analysis. Springer-Verlag. https://ggplot2.tidyverse.org
   Google Scholar
• Wilson, M., De Boeck, P., & Carstensen, C. H. (2008). Explanatory item response models: A brief introduction. In J. Hartig, E. Klieme, & D. Leutner (Eds.), Assessment of competencies in educational contexts (pp. 83–110). Hogrefe & Huber.
   Google Scholar
• Wood, K. E., Koenig, K., & Owens, L. (2018). Development of student abilities in control of variables at a two year college. AURCO Journal, 24, 164–179.
   Google Scholar
• Zhou, S., Han, J., Koenig, K., Raplinger, A., Pi, Y., Li, D., Xiao, H., Fu, Z., & Bao, L. (2016). Assessment of scientific reasoning : The effects of task context, data, and design on student reasoning in control of variables. Thinking Skills and Creativity, 19, 175–187. https://doi.org/10.1016/j.tsc.2015.11.004
   PubMed Web of Science ®Google Scholar
• Zwick, R., Thayer, D. T., & Lewis, C. (1999). An empirical Bayes approach to Mantel-Haenszel DIF analysis. Journal of Educational Measurement, 36(1), 1–28. https://doi.org/10.1111/j.1745-3984.1999.tb00543.x
   Web of Science ®Google Scholar