Block-based versus text-based programming environments on novice student learning outcomes: a meta-analysis study

References

• Altadmri, A., & Brown, N. C. C. (2015, March). 37 million compilations: Investigating novice programming mistakes in large-scale student data. In the ACM Technical Symposium on Computer Science Education (pp. 522–527). Kansas City, Missouri, USA: ACM.
   Google Scholar
• Bau, D., Gray, J., Kelleher, C., Sheldon, J., & Turbak, F. (2017, June). Learnable programming: Blocks and beyond. In the Communications of the ACM, 60(6), 72–80.
   Web of Science ®Google Scholar
• Bennedsen, J., & Caspersen, M. E. (2007). Failure rates in introductory programming. ACM SIGCSE Bulletin, 39(2), 32–36.
   Google Scholar
• Bloom, B. S., Englehart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1956). The taxonomy of educational objectives, handbook I: The cognitive domain. New York: David McKay Co., Inc.
   Google Scholar
• Borenstein, M., Hedges, L. V., Higgins, J. P., & Rothstein, H. R. (2011). Introduction to meta-analysis. Chichester, Hoboken: Wiley.
   Google Scholar
• Brown, N. C., Altadmri, A., & Kölling, M. (2016, March). Frame-based editing: Combining the best of blocks and text programming. In Learning and Teaching in Computing and Engineering (LaTICE), 2016 International Conference on, Mumbai (pp. 47–53). IEEE.
   Google Scholar
• Cilliers, C., Calitz, A., & Greyling, J. (2005, June). The effect of integrating an iconic programming notation into CS1. ACM SIGCSE Bulletin, 37(3), 108–112. *Studies included in the meta-analysis.
   Google Scholar
• Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155–159.
   PubMed Web of Science ®Google Scholar
• Cooper, H. (2017). Research synthesis and meta-analysis: A step-by-step approach (5th ed.). Thousand Oaks, CA: Sage.
   Google Scholar
• Costa, J. M., & Miranda, G. L. (2017). Relation between Alice software and programming learning: A systematic review of the literature and meta‐analysis. British Journal of Educational Technology, 48(6), 1464–1474.
   Web of Science ®Google Scholar
• CSEdWeek. (2018). Blurbs and useful stats. Retrieved from https://csedweek.org/resource_kit/blurbs
   Google Scholar
• Daly, T. (2011). Minimizing to maximize: An initial attempt at teaching introductory programming using Alice. Journal of Computing Sciences in Colleges, 26(5), 23–30. *Studies included in the meta-analysis.
   Google Scholar
• Duval, S., & Tweedie, R. (2000). Trim and fill: A simple funnel-plot–Based method of testing and adjusting for publication bias in meta-analysis. Biometrics, 56(2), 455–463.
   PubMed Web of Science ®Google Scholar
• EdSurge. (2017). Google, Facebook, Amazon among tech titans committing $300 million to K-12 computer science. Retrieved from https://www.edsurge.com/news/2017-09-26-google-facebook-amazon-among-tech-titans-committing-300-million-to-k-12-computer-science
   Google Scholar
• Eshach, H. (2007). Bridging in-school and out-of-school learning: Formal, non-formal, and informal education. Journal of Science Education and Technology, 16(2), 171–190.
   Google Scholar
• Fesakis, G., & Serafeim, K. (2009). Influence of the familiarization with scratch on future teachers’ opinions and attitudes about programming and ICT in education. ACM SIGCSE Bulletin, 41(3), 258–262.
   Google Scholar
• Garlick, R., & Cankaya, E. C. (2010, June). Using alice in CS1: A quantitative experiment. In Proceedings of the fifteenth annual conference on Innovation and technology in computer science education, Bilkent, Ankara, Turkey (pp. 165–168). ACM. *Studies included in the meta-analysis.
   Google Scholar
• Greenfield, P., & Lave, J. (1982). Cognitive aspects of informal education. In D. Wagner & H. Stevenson (Eds.), Cultural perspectives on child development (pp. 181–207). San Francisco, CA: W. H. Freeman.
   Google Scholar
• Grover, S., Basu, S., Bienkowski, M., Eagle, M., Diana, N., & Stamper, J. (2017). A framework for using hypothesis-driven approaches to support data-driven learning analytics in measuring computational thinking in block-based programming environments. ACM Transactions on Computing Education (TOCE), 17(3), 14:1–14: 25.
   Web of Science ®Google Scholar
• Gul, S., Asif, M., Ahmad, W., & Ahmad, U. (2017, December). Teaching programming: A mind map based methodology to improve learning outcomes. In Information and Communication Technologies (ICICT), 2017 International Conference on, Karachi (pp. 209–213). IEEE. *Studies included in the meta-analysis.
   Google Scholar
• Halverson, E. R., & Sheridan, K. (2014). The maker movement in education. Harvard Educational Review, 84(4), 495–504.
   Web of Science ®Google Scholar
• Harel, I., & Papert, S. (Eds.). (1991). Constructionism. Westport, CT: Ablex Publishing.
   Google Scholar
• Higgins, J., & Thompson, S. G. (2002). Quantifying heterogeneity in a meta-analysis. Statistics in Medicine, 21(11), 1539–1558.
   PubMed Web of Science ®Google Scholar
• Kelleher, C., & Pausch, R. (2005). Lowering the barriers to programming: A taxonomy of programming environments and languages for novice programmers. ACM Computing Surveys, 37(2), 83–137.
   Web of Science ®Google Scholar
• King, A. (1993). From sage on the stage to guide on the side. College Teaching, 41(1), 30–35.
   Google Scholar
• Krathwohl, D. R., Bloom, B. S., & Masia, B. B. (1964). Taxonomy of educational objectives, the classification of educational goals, handbook II: Affective domain. New York. David McKay Co., Inc.
   Google Scholar
• Lahtinen, E., Ala-Mutka, K., & Järvinen, H. M. (2005). A study of the difficulties of novice programmers. ACM SIGCSE Bulletin, 37(3), 14–18.
   Google Scholar
• Lewis, C. M. (2010, March). How programming environment shapes perception, learning and goals: Logo vs. scratch. In Proceedings of the 41st ACM technical symposium on Computer science education, Milwaukee, Wisconsin(pp. 346–350). ACM. *Studies included in the meta-analysis.
   Google Scholar
• Lin, J. M. C., & Yang, M. C. (2009). Analyzing student performance and attitudes toward textual versus iconic programming languages. Journal of Computers in Mathematics and Science Teaching, 28(1), 71–89. *Studies included in the meta-analysis.
   Google Scholar
• Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. Thousand Oaks, CA: Sage.
   Google Scholar
• Maarschalk, J. (1988). Scientific literacy and informal science teaching. Journal of Research in Science Teaching, 25(2), 135–146.
   Web of Science ®Google Scholar
• Meerbaum-Salant, O., Armoni, M., & Ben-Ari, M. (2011, June). Habits of programming in scratch. In Proceedings of the 16th annual joint conference on Innovation and technology in computer science education, Darmstadt, Germany (pp. 168–172). ACM.
   Google Scholar
• Mladenović, M., Boljat, I., & Žanko, Ž. (2017). Comparing loops misconceptions in block-based and text-based programming languages at the K-12 level. Education and Information Technologies, 23(44), 1483–1500. *Studies included in the meta-analysis.
   Google Scholar
• Mladenović, M., Rosić, M., & Mladenović, S. (2016). Comparing elementary students’ programming success based on programming environment. International Journal of Modern Education and Computer Science, 8(8), 1–10. *Studies included in the meta-analysis.
   Google Scholar
• Noone, M., & Mooney, A. (2018). Visual and textual programming languages: A systematic review of the literature. Journal of Computers in Education, 5(2), 149–174.
   Web of Science ®Google Scholar
• Obama White House Archives. (2016). Computer science for all. Retrieved from https://obamawhitehouse.archives.gov/blog/2016/01/30/computer-science-all
   Google Scholar
• Okita, S. Y. (2014). The relative merits of transparency: Investigating situations that support the use of robotics in developing student learning adaptability across virtual and physical computing platforms. British Journal of Educational Technology, 45(5), 844–862. *Studies included in the meta-analysis.
   Web of Science ®Google Scholar
• Orwin, R. G. (1983). A fail-safe N for effect size in meta-analysis. Journal of Educational Statistics, 8(2), 157–159.
   Google Scholar
• Papert, S. (1980). Mindstorms—Children, computers and powerful ideas. New York: Basic Books, Inc.
   Google Scholar
• Price, T. W., & Barnes, T. (2015, July). Comparing textual and block interfaces in a novice programming environment. In Proceedings of the eleventh annual International Conference on International Computing Education Research, Omaha, Nebraska (pp. 91–99). ACM. *Studies included in the meta-analysis.
   Google Scholar
• Reed, H. B., & Loughran, E. L. (1984). Beyond schools: Education for economic, social and personal development. Publications, CITP/CERC,225 Furcolo Hall. Amherst, MA: University of Massachusetts.
   Google Scholar
• Resnick, M., Maloney, J., Monroy-Hernández, A., Rusk, N., Eastmond, E., Brennan, K., … Kafai, Y. (2009). Scratch: Programming for all. Communications of the ACM, 52(11), 60–67.
   Web of Science ®Google Scholar
• Resnick, M., Ocko, S., & Papert, S. (1988). LEGO, Logo, and design. Children’s Environments Quarterly, 5(4), 14–18.
   Google Scholar
• Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86(3), 638–641.
   Web of Science ®Google Scholar
• Rosenthal, R. (1995). Writing meta-analytic reviews. Psychological Bulletin, 118(2), 183–192.
   Web of Science ®Google Scholar
• Ruf, A., Mühling, A., & Hubwieser, P. (2014, November). Scratch vs. Karel: Impact on learning outcomes and motivation. In Proceedings of the 9th Workshop in Primary and Secondary Computing Education, Berlin, Germany (pp. 50–59). ACM. *Studies included in the meta-analysis.
   Google Scholar
• Saito, D., Washizaki, H., & Fukazawa, Y. (2016, December). Analysis of the learning effects between text-based and visual-based beginner programming environments. In Engineering Education (ICEED), 2016 IEEE 8th International Conference on, Kuala Lumpur (pp. 208–213). IEEE. *Studies included in the meta-analysis.
   Google Scholar
• Saito, D., Washizaki, H., & Fukazawa, Y. (2017). Comparison of text-based and visual-based programming input methods for first-time learners. Journal of Information Technology Education-Research, 16, 209–226. *Studies included in the meta-analysis.
   Web of Science ®Google Scholar
• Shute, V. J. (2011). Stealth assessment in computer-based games to support learning. Computer Games and Instruction, 55(2), 503–524.
   Google Scholar
• Smith, M. (1988). Developing youth work: Informal education, mutual aid and popular practice. Milton Keynes, UK: Open University Press.
   Google Scholar
• Smith, M. (2006). Beyond the curriculum: Fostering associational life in schools. In Z. Bekerman, N. Burbules, & D. Silberman-Keller (Eds.), Learning in places: The informal education reader (pp. 9–33). New York: Peter Lang.
   Google Scholar
• Sterne, J. A., & Egger, M. (2001). Funnel plots for detecting bias in meta-analysis: Guidelines on choice of axis. Journal of Clinical Epidemiology, 54(10), 1046–1055.
   PubMed Web of Science ®Google Scholar
• Umapathy, K., & Ritzhaupt, A. D. (2017). A meta-analysis of pair-programming in computer programming courses: Implications for educational practice. ACM Transactions on Computing Education (TOCE), 17(4), 16.
   Web of Science ®Google Scholar
• Vaca-Cárdenas, L. A., Bertacchini, F., Tavernise, A., Gabriele, L., Valenti, A., Olmedo, D. E., … Bilotta, E. (2015, September). Coding with Scratch: The design of an educational setting for elementary pre-service teachers. In Interactive Collaborative Learning (ICL), 2015 International Conference on, Florence (pp. 1171–1177). IEEE.
   Google Scholar
• Vadeboncoeur, J. (2006). Engaging young people: Learning in informal contexts. Review of Research in Education, 30, 239–278.
   Web of Science ®Google Scholar
• Watson, C., & Li, F. W. (2014, June). Failure rates in introductory programming revisited. In Proceedings of the 2014 conference on Innovation & technology in computer science education, Uppsala, Sweden (pp. 39–44). ACM.
   Google Scholar
• Weintrop, D., Killen, H., & Franke, B. (2018). Blocks or text? How programming language modality makes a difference in assessing underrepresented populations. In J. Kay & R. Luckin (Eds.). Rethinking Learning in the Digital Age: Making the Learning Sciences Count, 13th International Conference of the Learning Sciences (ICLS) 2018. London, UK.
   Google Scholar
• Weintrop, D., Bain, C., & Wilensky, U. (n.d.). Blocking progress? Transitioning from block-based to text-based programming. *Studies included in the meta-analysis. Retrieved from https://ccl.northwestern.edu/2017/BlockingProgress.pdf
   Google Scholar
• Weintrop, D., & Wilensky, U. (2015, July). Using commutative assessments to compare conceptual understanding in blocks-based and text-based programs. In ICER, Omaha, NE (Vol. 15, pp. 101–110). *Studies included in the meta-analysis.
   Google Scholar
• Weintrop, D., & Wilensky, U. (2017a). Between a block and a typeface: Designing and evaluating hybrid programming environments. In Proceedings of the 2017 Conference on Interaction Design and Children, Stanford, California (pp. 183–192). ACM. *Studies included in the meta-analysis.
   Google Scholar
• Weintrop, D., & Wilensky, U. (2017b). Comparing block-based and text-based programming in high school computer science classrooms. ACM Transactions on Computing Education (TOCE), 18(1), 1–25. *Studies included in the meta-analysis.
   Web of Science ®Google Scholar
• Weintrop, D., & Wilensky, U. (2018). How block-based, text-based, and hybrid block/text modalities shape novice programming practices. International Journal of Child-Computer Interaction, 17, 83–92.
   Google Scholar
• Wing, J. M. (2006). Computational thinking. Communications of the ACM, 49(3), 33–35. doi:10.1145/1118178
   Web of Science ®Google Scholar
• Wing, J. M. (2008). Computational thinking and thinking about computing. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 366(1881), 3717–3725. doi:10.1098/rsta.2008.0118
   PubMed Web of Science ®Google Scholar