Reevaluating the relationship between explaining, tracing, and writing skills in CS1 in a replication study

References

• Andrich, D. (1978). A rating formulation for ordered response categories. Psychometrika, 43(4), 561–573. https://doi.org/https://doi.org/10.1007/BF02293814
   Web of Science ®Google Scholar
• Bollen, K. A. (1989). Structural equations with latent variables. Wiley.
   Google Scholar
• Chen, Z., Chudzicki, C., Palumbo, D., Alexandron, G., Choi, Y.-J., Zhou, Q., & Pritchard, D. E. (2016, April). Researching for better instructional methods using AB experiments in MOOCs: Results and challenges. Research and Practice in Technology Enhanced Learning, 11(1), 9. Retrieved December 30, 2021 from https://doi.org/https://doi.org/10.1186/s41039-016-0034-4
   PubMedGoogle Scholar
• Chen, B., Zilles, C., West, M., & Bretl, T. (2019). Effect of discrete and continuous parameter variation on difficulty in automatic item generation. In S. Isotani, E. Millán, A. Ogan, P. Hastings, B. McLaren, & R. Luckin, (Eds.), Artificial intelligence in education (pp. 11625). Lecture Notes in Computer Science, Springer, Cham. https://doi.org/https://doi.org/10.1007/978-3-030-23204-7-7
   Google Scholar
• Clear, T. (2005). Comprehending large code bases-the skills required for working in a” brown fields” environment. ACM SIGCSE Bulletin, 37(2), 12–14. https://doi.org/https://doi.org/10.1145/1083431.1083439
   Google Scholar
• Clear, T., Whalley, J., Robbins, P., Philpott, A., Eckerdal, A., & Laakso, M.-J. (2011 ()). Report on the final bracelet workshop: Auckland university of technology. CITRENZ, September 2010.
   Google Scholar
• Corbett, A. T., & Anderson, J. R. (2001). Locus of feedback control in computer-based tutoring: Impact on learning rate, achievement and attitudes. In Proceedings of the Sigchi conference on human factors in computing systems (pp. 245–252). New York, NY, USA: Association for Computing Machinery. https://doi.org/https://doi.org/10.1145/365024.365111
   Google Scholar
• Corley, J., Stanescu, A., Baumstark, L., & Orsega, M. C. (2020). Paper or ide? the im- pact of exam format on student performance in a cs1 course. In Proceedings of the 51st ACM technical symposium on computer science education (pp. 706–712). New York, NY, USA: Association for Computing Machinery. https://doi.org/https://doi.org/10.1145/3328778.3366857
   Google Scholar
• Corney, M., Fitzgerald, S., Hanks, B., Lister, R., McCauley, R., & Murphy, L. (2014). ‘explain in plain English’ questions revisited: Data structures problems. In Proceedings of the 45th ACM technical symposium on computer science education (pp. 591–596). New York, NY, USA: Association for Computing Machinery. https://doi.org/https://doi.org/10.1145/2538862.2538911
   Google Scholar
• Cunningham, K., Blanchard, S., Ericson, B., & Guzdial, M. (2017). Using tracing and sketching to solve programming problems: Replicating and extending an analysis of what students draw. In Proceedings of the 2017 ACM conference on international computing education re-search (pp. 164–172). New York, NY, USA: Association for Computing Machinery. https://doi.org/https://doi.org/10.1145/3105726.3106190
   Google Scholar
• Cunningham, K., Ke, S., Guzdial, M., & Ericson, B. (2019). Novice rationales for sketching and tracing, and how they try to avoid it. In Proceedings of the 2019 ACM conference on innovation and technology in computer science education (pp. 37–43). New York, NY, USA: Association for Computing Machinery. https://doi.org/https://doi.org/10.1145/3304221.3319788
   Google Scholar
• Fan, Y., Chen, J., Shirkey, G., John, R., Wu, S. R., Park, H., & Shao, C. (2016, November). Applications of structural equation modeling (SEM) in ecological studies: An updated review. Ecological Processes, 5(1), 19. Retrieved December 28, 2021 from https://doi.org/https://doi.org/10.1186/s13717-016-0063-3
   Google Scholar
• Fowler, M., Chen, B., & Zilles, C. (2021). How should we ‘explain in plain English’? voices from the community. In Proceedings of the 17th ACM conference on international computing education research (pp. 69–80). New York, NY, USA: Association for Computing Machinery. https://doi.org/https://doi.org/10.1145/3446871.3469738
   Google Scholar
• Gierl, M., & Haladyna, T. (Eds.). (2012). Automatic item generation: Theory and practice (1st ed.). Routledge. https://doi.org/https://doi.org/10.4324/9780203803912
   Google Scholar
• Grissom, S., Murphy, L., McCauley, R., & Fitzgerald, S. (2016). Paper vs. computer- based exams: A study of errors in recursive binary tree algorithms. In Proceedings of the 47th ACM technical symposium on computing science education (pp. 6–11). New York, NY, USA: Association for Computing Machinery. https://doi.org/https://doi.org/10.1145/2839509.2844587
   Google Scholar
• Guay, R. B., & McCabe, G. P. (1986). A binomial test for hierarchical dependency. Psychometrika, 51(3), 467–474. https://doi.org/http://doi.org/10.1007/BF02294067
   Google Scholar
• Harrington, B., & Cheng, N. (2018). Tracing vs. writing code: Beyond the learning hierarchy. In Proceedings of the 49th ACM technical symposium on computer science education (pp. 423–428).
   Google Scholar
• Hassan, M., & Zilles, C. (2021). Exploring ‘reverse-tracing’questions as a means of assessing the tracing skill on computer-based CS 1 exams. In Proceedings of the 17th ACM conference on international computing education research (pp. 115–126).
   Google Scholar
• Haughton, D. M., Oud, J. H., & Jansen, R. A. (1997). Information and other criteria in structural equation model selection. Communications in Statistics-Simulation and Computation, 26(4), 1477–1516. https://doi.org/https://doi.org/10.1080/03610919708813451
   Web of Science ®Google Scholar
• Jarque, C. M., & Bera, A. K. (1980). Efficient tests for normality, homoscedasticity and serial Independence of regression residuals. Economics Letters, 6(3), 255–259. https://doi.org/https://doi.org/10.1016/0165-1765(80)90024-5
   Web of Science ®Google Scholar
• Kikuchi, S., & Hamamoto, K. (2016). Investigating the relationship between tracing skill and modification skill for different programming statements. Proceedings of the School of Information and Telecommunication Engineering, 9( 1), 8–14. Tokai University.
   Google Scholar
• Koedinger, K. R., Booth, J. L., & Klahr, D. (2013). Instructional complexity and the science to constrain it. Science, 342(6161), 935–937. https://doi.org/https://doi.org/10.1126/science.1238056
   PubMed Web of Science ®Google Scholar
• Kumar, A. N. (2013). A study of the influence of code-tracing problems on code-writing skills. In Proceedings of the 18th ACM conference on innovation and technology in computer science education (pp. 183–188).
   Google Scholar
• Kumar, A. N. (2015). Solving code-tracing problems and its effect on code-writing skills pertaining to program semantics. In Proceedings of the 2015 ACM conference on innovation and technology in computer science education (pp. 314–319).
   Google Scholar
• Kurland, D. M., Pea, R. D., Clement, C., & Mawby, R. (1986). A study of the development of programming ability and thinking skills in high school students. Journal of Educational Computing Research, 2(4), 429–458. https://doi.org/https://doi.org/10.2190/BKML-B1QV-KDN4-8ULH
   Google Scholar
• Lappalainen, V., Lakanen, A.-J., & Högmander, H. (2017). Towards computer-based exams in CS1. In Proceedings of the 9th international conference on computer supported education. SCITEPRESS Science and Technology Publications.
   Google Scholar
• Lin, L.-C., Huang, P.-H., & Weng, L.-J. (2017). Selecting path models in sem: A comparison of model selection criteria. Structural Equation Modeling: A Multidisciplinary Journal, 24(6), 855–869. https://doi.org/https://doi.org/10.1080/10705511.2017.1363652
   Web of Science ®Google Scholar
• Lister, R., Adams, E. S., Fitzgerald, S., Fone, W., Hamer, J., Lindholm, M., McCartney, R., Moström, J. E., Sanders, K., Seppälä, O., Simon, B., Thomas, L. (2004). A multi-national study of reading and tracing skills in novice programmers. ACM SIGCSE Bulletin, 36(4), 119–150. https://doi.org/https://doi.org/10.1145/1041624.1041673
   Google Scholar
• Lister, R., Simon, B., Thompson, E., Whalley, J. L., & Prasad, C. (2006). Not seeing the forest for the trees: Novice programmers and the solo taxonomy. ACM SIGCSE Bulletin, 38(3), 118–122. https://doi.org/https://doi.org/10.1145/1140123.1140157
   Google Scholar
• Lister, R., Fidge, C., & Teague, D. (2009). Further evidence of a relationship between explain- ing, tracing and writing skills in introductory programming. ACM SIGCSE Bulletin, 41(3), 161–165. https://doi.org/https://doi.org/10.1145/1595496.1562930
   Google Scholar
• Lister, R. (2020). On the cognitive development of the novice programmer: And the development of a computing education researcher. In Proceedings of the 9th computer science education research conference. New York, NY, USA: Association for Computing Machinery. https://doi.org/https://doi.org/10.1145/3442481.3442498
   Google Scholar
• Lopez, M., Whalley, J., Robbins, P., & Lister, R. (2008). Relationships between reading, tracing and writing skills in introductory programming. In Proceedings of the fourth international workshop on computing education research (pp. 101–112).
   Google Scholar
• McCracken, M., Almstrum, V., Diaz, D., Guzdial, M., Hagan, D., Kolikant, Y. B.-D., … Wilusz, T. (2001). A multi-national, multi-institutional study of assessment of programming skills of first-year cs students. In Working group reports from iticse on innovation and technology in computer science education (pp. 125–180).
   Google Scholar
• Mendoza, B., & Zavala, L. (2018). An intervention strategy to hone students’ code under- standing skills. Journal of Computing Sciences in Colleges, 33(3), 105–114. https://doi.org/http://doi.org/10.5555/3144687.3144714
   Google Scholar
• Moshagen, M. (2020). sempower: Power analyses for sem [Computer software manual]. (R package version 1.1.0) https://CRAN.R-project.org/package=semPower
   Google Scholar
• Moubayed, A., Injadat, M., Shami, A., & Lutfiyya, H. (2018, March). Relationship between student engagement and performance in e-learning environment using association rules. In 2018 IEEE World Engineering Education Conference (EDUNINE) (pp. 1–6). Retrieved December 28, 2021, from http://arxiv.org/abs/2101.02006 (arXiv: 2101.02006)
   Google Scholar
• Murphy, L., McCauley, R., & Fitzgerald, S. (2012). ‘explain in plain English’questions: Im- plications for teaching. In Proceedings of the 43rd ACM technical symposium on computer science education (pp. 385–390).
   Google Scholar
• Murphy, L., Fitzgerald, S., Lister, R., & McCauley, R. (2012). Ability to’explain in plain English’linked to proficiency in computer-based programming. In Proceedings of the ninth annual international conference on international computing education research (pp. 111–118).
   Google Scholar
• Parsons, D., & Haden, P. (2006). Parson’s programming puzzles: A fun and effective learning tool for first programming courses. Proceedings of the 8th Australasian conference on computing education, 52, 157–163.
   Google Scholar
• Pelchen, T., Mathieson, L., & Lister, R. (2020). On the evidence for a learning hierarchy in data structures exams. In Proceedings of the twenty-second Australasian computing education conference (pp. 122–131).
   Google Scholar
• Perkins, D., & Martin, F. (1986). Fragile knowledge and neglected strategies in novice programmers. In At empirical studies of programmers, 1st workshop (pp. 213–229). Washington, DC: Ablex Publishing Corp.
   Google Scholar
• Preacher, K. J., & Merkle, E. C. (2012). The problem of model selection uncertainty in structural equation modeling. Psychological Methods, 17(1), 1. https://doi.org/https://doi.org/10.1037/a0026804
   PubMed Web of Science ®Google Scholar
• R Core Team. (2020). R: A language and environment for statistical computing [Computer software manual]. https://www.R-project.org/
   Google Scholar
• Rasch, G. (1993). Probabilistic models for some intelligence and attainment tests. ERIC.
   Google Scholar
• Royle, G., Mckay, B., Oggier, F., Sloane, N., Wanless, I., & Wilf, H. (2004). Acyclic digraphs and eigenvalues of (0, 1)-matrices. Journal of Integer Sequences, 7(3), online–approx. https://doi.org/https://doi.org/10.48550/arXiv.math/0310423
   Google Scholar
• Schooler, L. J., & Anderson, J. R. (1990). The disruptive potential of immediate feedback. Proceedings of the Twelfth Annual Conference of the Cognitive Science Society, Cambridge, MA.
   Google Scholar
• Sheard, J., Carbone, A., Lister, R., Simon, B., Thompson, E., & Whalley, J. L. (2008). Going solo to assess novice programmers. In Proceedings of the 13th annual conference on innovation and technology in computer science education (pp. 209–213).
   Google Scholar
• Simon, B., Lopez, M., Sutton, K., & Clear, T. (2009). Surely we must learn to read before we learn to write! Proceedings of the eleventh Australasian conference on computing education, 95, 165–170.
   Google Scholar
• Soloway, E., Bonar, J., & Ehrlich, K. (1983). Cognitive strategies and looping constructs: An empirical study. Communications of the ACM, 26(11), 853–860. https://doi.org/https://doi.org/10.1145/182.358436
   Web of Science ®Google Scholar
• Soloway, E. (1986, September). Learning to program = learning to construct mechanisms and explanations. Communications of the ACM, 29(9), 850–858. https://doi.org/http://doi.org/10.1145/6592.6594
   Web of Science ®Google Scholar
• Soloway, E., & Spohrer, J. C. (1989). Studying the novice programmer. Psychology Press.
   Google Scholar
• Sudol-DeLyser, L. A. (2015). Expression of abstraction: Self explanation in code production. In Proceedings of the 46th ACM technical symposium on computer science education (pp. 272–277).
   Google Scholar
• Tan, G., & Venables, A. (2010). Wearing the assessment ‘bracelet’. Journal of Information Technology Education: Innovations in Practice, 9(1), 25–34. https://doi.org/http://doi.org/10.28945/707
   Google Scholar
• Teague, D. M., Corney, M. W., Ahadi, A., & Lister, R. (2012). Swapping as the” hello world” of relational reasoning: Replications, reflections and extensions. Proceedings of conferences in research and practice in information technology (crpit), 123.
   Google Scholar
• Vainio, V., & Sajaniemi, J. (2007). Factors in novice programmers’ poor tracing skills. In Proceedings of the 12th annual Sigcse conference on innovation and technology in computer science education (pp. 236–240). New York, NY, USA: Association for Computing Machinery. https://doi.org/https://doi.org/10.1145/1268784.1268853
   Google Scholar
• Venables, A., Tan, G., & Lister, R. (2009). A closer look at tracing, explaining and code writing skills in the novice programmer. In Proceedings of the fifth international workshop on computing education research workshop (pp. 117–128).
   Google Scholar
• Weinman, N., Fox, A., & Hearst, M. A. (2021). Improving instruction of programming patterns with faded parsons problems. In Proceedings of the 2021 chi conference on human factors in computing systems. New York, NY, USA: Association for Computing Machinery. https://doi.org/https://doi.org/10.1145/3411764.3445228
   Google Scholar
• West, M., Herman, G. L., & Zilles, C. (2015). PrairieLearn: Mastery-based online problem solving with adaptive scoring and recommendations driven by machine learning. Paper presented at the ASEE Annual Conference & Exposition, ASEE Conferences, Seatttle, Washington.
   Google Scholar
• Whalley, J. L., Lister, R., Thompson, E., Clear, T., Robbins, P., Kumar, P., & Prasad, C. (2006). An Australasian study of reading and comprehension skills in novice programmers, using the bloom and solo taxonomies. Proceedings of the 8th Australasian conference on computing education, 52, 243–252.
   Google Scholar
• Winslow, L. E. (1996). Programming pedagogy—a psychological overview. ACM Sigcse Bulletin, 28(3), 17–22. https://doi.org/https://doi.org/10.1145/234867.234872
   Google Scholar
• Wolf, E. J., Harrington, K. M., Clark, S. L., & Miller, M. W. (2013, December). Sample size requirements for structural equation models: An evaluation of power, bias, and solution propriety. Educational and Psychological Measurement, 76(6), 913–934. (25705052[pmid]) https://pubmed.ncbi.nlm.nih.gov/25705052
   PubMed Web of Science ®Google Scholar
• Xie, B., Nelson, G. L., & Ko, A. J. (2018). An explicit strategy to scaffold novice program tracing. In Proceedings of the 49th ACM technical symposium on computer science education (pp. 344–349). New York, NY, USA: Association for Computing Machinery. https://doi.org/https://doi.org/10.1145/3159450.3159527
   Google Scholar
• Xie, B., Loksa, D., Nelson, G. L., Davidson, M. J., Dong, D., Kwik, H., Tan, A. H., Hwa, L., Li, M., & Ko, A. J. (2019). A theory of instruction for introductory programming skills. Computer Science Education, 29(2–3), 205–253. https://doi.org/https://doi.org/10.1080/08993408.2019.1565235
   Web of Science ®Google Scholar
• Yamamoto, M., Sekiya, T., & Yamaguchi, K. (2011). Relationship between programming concepts underlying programming skills. In 2011 international conference on information technology based higher education and training (pp. 1–7).
   Google Scholar
• Yamamoto, M., Sekiya, T., Mori, K., & Yamaguchi, K. (2012). Skill hierarchy revised by sem and additional skills. In 2012 international conference on information technology based higher education and training (ithet) (pp. 1–8).
   Google Scholar