Coding Code: Qualitative Methods for Investigating Data Science Skills

References

• Arnold, P., Confrey, J., Jones, R. S., Lee, H. S., and Pfannkuch, M. (2018), “Statistics Learning Trajectories,” in International Handbook of Research in Statistics Education, eds. B. Ben-Zvi, K. Ben-Zvi, and J. Garfield, pp. 295–326, Cham: Springer.
   Google Scholar
• Beckman, M. D., Cetinkaya-Rundel, M., Horton, N. J., Rundel, C., Sullivan, A. J., and Tackett, M. (2021), “Implementing Version Control with Git and GitHub as a Learning Objective in Statistics and Data Science Courses,” Journal of Statistics and Data Science Education, 29, S132–S144. DOI: 10.1080/10691898.2020.1848485.
   Web of Science ®Google Scholar
• Biehler, R. (1997), “Software for Learning and for Doing Statistics,” International Statistics Review, 65, 167–189. DOI: 10.2307/1403342.
   Web of Science ®Google Scholar
• Broatch, J. E., Dietrich, S., and Goelman, D. (2019), “Introducing Data Science Techniques by Connecting Database Concepts and dplyr,” Journal of Statistics and Data Science Education, 27, 147–153. DOI: 10.1080/10691898.2019.1647768.
   Web of Science ®Google Scholar
• Burr, W., Chevalier, F., Collins, C., Gibbs, A., Ng, R., and Wild, C. (2021), “Computational Skills by Stealth in Introductory Data Science Teaching,” Teaching Statistics, 43, S34–S51. DOI: 10.1111/test.12277.
   Web of Science ®Google Scholar
• Çetinkaya-Rundel, M., and Ellison, V. (2020), “A Fresh Look at Introductory Data Science,” Journal of Statistics and Data Science Education, 29, S17–S26. DOI: 10.1080/10691898.2020.1804497.
   Web of Science ®Google Scholar
• Çetinkaya-Rundel, M., Hardin, J. S., Baumer, B. S., McNamara, A., Horton, N. J., and Rundel, C. W. (2022), “An Educator’s Perspective of the tidyverse,” Technology Innovations in Statistics Education, 14. DOI: 10.5070/T514154352.
   Google Scholar
• Cobb, P., Confrey, J., diSessa, A., Lehrer, R., and Schauble, L. (2003), “Design Experiments in Educational Research,” Educational Researcher, 32, 9–13. DOI: 10.3102/0013189X0320010.
   Google Scholar
• Confrey, J. (2006), “The Evolution of Design Studies as Methodology,” in Cambridge Handbook of the Learning Sciences, ed. K. Swayer, pp. 131–151, Cambridge: Cambridge University Press. DOI: 10.1017/CBO9780511816833.010.
   Google Scholar
• Confrey, J., and Lachance, A. (2000), “Transformative Teaching Experiments Through Conjecture-driven Research Design,” in Handbook of Research Design in Mathematics and Science Education, eds. A. Kelly and R. Lesh, pp. 231–265, Mahwah, NJ: Lawrence Erlbaum Associates. DOI: 10.4324/9781410602725.ch10.
   Google Scholar
• Corbin, J., and Strauss, A. (2008), Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory, Thousand Oaks, CA: Sage.
   Google Scholar
• Creswell, J. W., and Poth, C. N. (2018), Qualitative Inquiry & Research Design, Thousand Oaks, CA: Sage.
   Google Scholar
• Danyluk, A., Leidig, P., Buck, S., Cassel, L., Doyle, M., Ho, T. K., McGettrick, A., McIntosh, S., Qian, W., Schmitt, K., Servin, C., Stefik, A., Wang, H., and Wittenbach, J. (2019), “ACM Data Science Task Force Draft Report,” available at https://dstf.acm.org/DSTF_Final_Report.pdf.
   Google Scholar
• Dey, I. (1993), Qualitative Data Analysis: A User-friendly Guide for Social Scientists, London: Routledge.
   Google Scholar
• Fergusson, A., and Pfannkuch, M. (2021), “Introducing Teachers Who Use GUI-Driven Tools for the Randomization Test to Code-Driven Tools,” Mathematical Thinking and Learning, 24, 336–356. DOI: 10.1080/10986065.2021.1922856.
   Web of Science ®Google Scholar
• Findley, K. (2022), “Navigating a Disciplinary Chasm: The Statistical Perspectives of Graduate Teaching Assistants,” Statistics Education Research Journal, 21, Article 12. DOI: 10.52041/serj.v21i1.14.
   Google Scholar
• Gehrke, M., Kistler, T., Lubke, K., Markgraf, N., Krol, B., and Sauer, S. (2021), “Statistics Education from a Data-centric Perspective,” Teaching Statistics, 43, S201–S215. DOI: 10.1111/test.12264.
   Web of Science ®Google Scholar
• Gravemeijer, K., and Cobb, P. (2006), “Design Research from a Learning Design Perspective,” in Educational Design Research, ed. J. van den Akker, pp. 17–51, London: Routledge.
   Google Scholar
• Groth, R. E. (2010), “Interactions among Knowledge, Beliefs, and Goals in Framing a Qualitative Study in Statistics Education,” Journal of Statistics Education, 18. DOI: 10.1080/10691898.2010.11889475.
   PubMedGoogle Scholar
• Guzman, L. M., Pennell, M. W., Nikelski, E., and Srivastava, D. S. (2019), “Successful Integration of Data Science in Undergraduate Biostatistics Courses Using Cognitive Load Theory,” CBE–Life Sciences Education, 18, 1–10. DOI: 10.1187/cbe.19-02-0041.
   Web of Science ®Google Scholar
• Izu, C., Schulte, C., Aggarwal, A., Cutts, Q., Duran, R., Gutica, M., Heinemann, B., Kraemer, E., Lonati, V., Mirolo, C., and Weeda, R. (2019), “Fostering Program Comprehension in Novice Programmers – Learning Activities and Learning Trajectories,” in Proceedings of the Working Group Reports on Innovation and Technology in Computer Science Education (ITiCSE-WGR ’19). New York, NY: Association for Computing Machinery. DOI: 10.1145/3344429.3372501.
   Google Scholar
• Jadud, M. C. (2005), “A First Look at Novice Compilation Behavior Using BlueJ.” Computer Science Education, 15, 25–40. DOI: 10.1080/08993400500056530.
   Google Scholar
• Jadud, M. C. (2006), “Methods and Tools for Exploring Novice Compilation Behavior,” in Proceedings of the 2nd International Workshop on Computing Education Research (ICER), p. 73. Canterbury, UK: Association of Computing Machinery. DOI: 10.1145/1151588.1151600.
   Google Scholar
• Justice, N., Morris, S., Henry, V., and Brondos Fry, E. (2020), “Paint-by-Number or Picasso? A Grounded Theory Phenomenographical Study of Students’ Conceptions of Statistics,” Statistics Education Research Journal, 19, 76–102. DOI: 10.52041/serj.v19i2.111.
   Google Scholar
• Kross, S., and McGowan, D. (2020), matahari: Spy on Your R Session, 0.1.0 edition. Available at https://jhudatascience.org/matahari/.
   Google Scholar
• Lewis, C. M. (2012), “The Importance of Students’ Attention to Program State: A Case Study of Debugging Behavior,” in Proceedings of the 9th Annual International Conference on International Computing Education Research (ICER), pp. 127–134, New York, NY: Association of Computing Machinery. DOI: 10.1145/2361276.2361301.
   Google Scholar
• Lincoln, Y. S., and Guba, E. G. (1985), Naturalistic Inquiry, Thousand Oaks, CA: Sage.
   Google Scholar
• Lister, R., Adams, E. S., Fitzgerald, S., Fone, W., Hamer, J., Lindholm, M., McCartney, R., Moström, J. E., Sanders, K., Seppälä, O., Simon, B., and Thomas, L. (2004), “A Multi-National Study of Reading and Tracing Skills in Novice Programmers,” SIGCSE Bulletin, 36, 119–150. DOI: 10.1145/1041624.1041673.
   Google Scholar
• Lobato, J., and Walters, C. (2017), “A Taxonomy of Approaches to Learning Trajectories and Progressions,” in Compendium for Research in Mathematics Education, ed. J. Cai, pp. 74–101, Reston, VA: National Council of Teachers of Mathematics.
   Google Scholar
• Lovett, M. C., and Greenhouse, J. B. (2000), “Applying Cognitive Theory to Statistics Instruction,” The American Statistician, 54, 196–206. DOI: 10.2307/2685590.
   Web of Science ®Google Scholar
• Loy, A., Kuiper, S., and Chihara, L. (2019), “Supporting Data Science in the Statistics Curriculum,” Journal of Statistics Education, 27, 2–11. DOI: 10.1080/10691898.2018.1564638.
   Web of Science ®Google Scholar
• McCall, D., and Kolling, M. (2014), “Meaningful Categorisation of Novice Programer Errors,” in 2014 IEEE Frontiers in Education Conference (FIE) Proceedings, pp. 1–8, Madrid, Spain. DOI: 10.1109/FIE.2014.7044420.
   Google Scholar
• McNamara, A. (2015), “Bridging the Gap Between Tools for Learning and for Doing Statistics,” PhD thesis, University of California, Los Angeles.
   Google Scholar
• McNamara, A., Zieffler, A., Beckman, M., Legacy, C., Butler Basner, E., delMas, R., and Rao, V. V. (2021), “Computing in the Statistics Curriculum: Lessons Learned from the Educational Sciences,” in United States Conference on Teaching Statistics (USCOTS). Available at https://causeweb.org/cause/sites/default/files/uscots/uscots21/materials/Tu-03
   Google Scholar
• Merriam, S. B., and Tisdell, E. J. (2016), Qualitative Research, San Francisco, CA: Wiley.
   Google Scholar
• Miles, M. B., Huberman, A. M., and Saldaña, J. (2020), Qualitative Data Analysis, Thousand Oaks, CA: Sage.
   Google Scholar
• Myint, L., Hadavand, A., Jager, L., and Leek, J. (2020), “Comparison of Beginning R Students’ Perceptions of Peer-Made Plots Created in Two Plotting Systems: A Randomized Experiment,” Journal of Statistics Education, 1, 98–108. DOI: 10.1080/10691898.2019.1695554.
   Google Scholar
• National Academies of Sciences, Engineering, and Medicine. (2018), Data Science for Undergraduates: Opportunities and Options, Washington, DC: The National Academies Press. DOI: 10.17226/25104.
   Google Scholar
• Nolan, D., and Temple Lang, D. (2010), “Computing in the Statistics Curricula,” The American Statistician, 64, 97–107. DOI: 10.1198/tast.2010.09132.
   Web of Science ®Google Scholar
• Prediger, S., Gravemeijer, K., and Confrey, J. (2015), “Design Research with a Focus on Learning Processes – An Overview on Achievements and Challenges,” ZDM Mathematics Education, 47, 877–891. DOI: 10.1007/s11858-015-0722-3.
   Google Scholar
• Pruim, R., Kaplan, D. T., and Horton, N. J. (2017), “The mosaic Package: Helping Students to ‘Think with Data’ Using R,” The R Journal, 9, 77–102. DOI: 10.32614/RJ-2017-024.
   Google Scholar
• R Core Team. (2020). R: A Language and Environment for Statistical Computing, Vienna, Austria.
   Google Scholar
• Rafalski, T., Uesbeck, P. M., Panks-Meloney, C., Daleiden, P., Allee, W., McNamara, A., and Stefik, A. (2019), “A Randomized Controlled Trial on the Wild Wild West of Scientific Computing with Student Learners,” in Proceedings of the 2019 ACM Conference on International Computing Education Research, pp. 239–247. DOI: 10.1145/3291279.3339421.
   Google Scholar
• Reinhart, A., Evans, C., Luby, A., Orellana, J., Meyer, M., Wieczorek, J., Elliott, P., Burckhardt, P., and Nugent, R. (2022), “Think-Aloud Interviews: A Tool for Exploring Student Statistical Reasoning,” Journal of Statistics and Data Science Education, 30, 100–113. DOI: 10.1080/26939169.2022.2063209.
   Web of Science ®Google Scholar
• RStudio Team. (2020), RStudio: Integrated Development Environment for R, Boston, MA: RStudio PBR.
   Google Scholar
• Saldana, J. (2013), The Coding Manual for Qualitative Researchers, Thousand Oaks, CA: Sage.
   Google Scholar
• Schulte, C. (2008), “Block Model: An Educational Model of Program Comprehension as a Tool for a Scholarly Approach to Teaching,” in Proceedings of the Fourth International Workshop on Computing Education Research, pp. 149–160, Sydney, Australia: Association of Computing Machinery. DOI: 10.1145/1404520.1404535.
   Google Scholar
• Simon, A., Geldreich, K., and Hubwieser, P. (2019), “How to Transform Programming Processes in Scratch to Graphical Visualizations,” in Proceedings of the 14th Workshop in Primary and Secondary Computing Education, Glasgow, Scotland: Association of Computing Machinery. DOI: 10.1145/3361721.3361723.
   Google Scholar
• Spohrer, J. C., Soloway, E., and Pope, E. (1985), “A Goal/Plan Analysis of Buggy Pascal Programs,” Human-Computer Interaction, 1, 163–207. DOI: 10.1207/s15327051hci0102_4.
   Google Scholar
• Stefik, A., and Siebert, S. (2013), “An Empirical Investigation into Programming Language Syntax,” ACM Transactions on Computing Education, 13, 1–40. DOI: 10.1145/2534973.
   Web of Science ®Google Scholar
• Theobold, A. (2022), “Materials and Data Associated with Coding Code: Qualitative Methods for Investigating Data Science Skills,” DOI: 10.5281/ZENODO.7114764.
   Google Scholar
• Theobold, A. S. (2020), “Supporting Data-Intensive Environmental Science Research: Data Science Skills for Scientific Practitioners of Statistics,” PhD thesis. Montana State University, Department of Mathematical Sciences, Bozeman, Montana.
   Google Scholar
• Theobold, A. S., and Hancock, S. (2019), “How Environmental Science Graduate Students Acquire Statistical Computing Skills,” Statistics Education Research Journal, 18, 65–85. DOI: 10.52041/serj.v18i2.141.
   Google Scholar
• Weiland, T. (2019), “The Contextualized Situations Constructed for the Use of Statistics by School Mathematics Textbooks,” Statistics Education Research Journal, 18, 18–38. DOI: 10.52041/serj.v18i2.138.
   Google Scholar
• Wickham, H., Averick, M., Bryan, J., Chang, W., McGowan, L. D., François, R., Grolemund, G., Hayes, A., Henry, L., Hester, J., Kuhn, M., Pedersen, T. L., Miller, E., Bache, S. M., Müller, K., Ooms, J., Robinson, D., Seidel, D. P., Spinu, V., Takahashi, K., Vaughan, D., Wilke, C., Woo, K., and Yutani, H. (2019), “Welcome to the Tidyverse,” Journal of Open Source Software, 4, 1686. DOI: 10.21105/joss.01686.
   Google Scholar
• Wickham, H., and Grolemund, G. (2017), R for Data Science, Sebastopol, CA: O’Reilly.
   Google Scholar
• Wiggins, G., and McTighe, J. (2005), Understanding by Design (2nd ed.), Alexandria: Association for Supervision and Curriculum Development (ASCD).
   Google Scholar
• Wilson, G., Aruliah, D. A., Titus Brown, C., Chue Hong, N. P., Davis, M., Guy, R. T., Haddock, S. H. D., Huff, K. D., Mitchell, I. M., Plumbley, M. D., Waugh, B., White, E. P., and Wilson, P. (2014), “Best Practices for Scientific Computing,” PloS Biology, 12, e1001745. DOI: 10.1371/journal.pbio.1001745.
   PubMed Web of Science ®Google Scholar