Educating and Measuring Choice: A Test of the Transfer of Design Thinking in Problem Solving and Learning

References

• Acredolo, C., & Horobin, K. (1987). Development of relational reasoning and avoidance of premature closure. Developmental Psychology, 23(1), 13. doi:10.1037/0012-1649.23.1.13
   Web of Science ®Google Scholar
• Ammons, R. B. (1956). Effects of knowledge of performance: A survey and tentative theoretical formulation. Journal of General Psychology, 54, 279–299. doi:10.1080/00221309.1956.9920284
   Web of Science ®Google Scholar
• Baker, R. S., Corbett, A. T., Roll, I., & Koedinger, K. R. (2008). Developing a generalizable detector of when students game the system. User Modeling and User-Adapted Interaction, 18(3), 287–314. doi:10.1007/s11257-007-9045-6
   Web of Science ®Google Scholar
• Baker, R. S., & Corbett, A. T., Koedinger, K. R. (2004). Detecting student misuse of intelligent tutoring systems. In J. C. Lester, R. M. Vicari, & F. Paraguaçu (Eds.), Lecture notes in computer science, vol. 3220: Intelligent Tutoring Systems (pp. 531–540). Heidelberg, Germany: Springer.
   Google Scholar
• Bamberger, Y., & Cahill, C. (2013). Teaching design in middle-school: Instructors’ concerns and scaffolding strategies. Journal of Science Education and Technology, 22(2), 171–185. doi:10.1007/s10956-012-9384-x
   Web of Science ®Google Scholar
• Barron, B. J., Schwartz, D. L., Vye, N. J., Moore, A., Petrosino, A., Zech, L., & Bransford, J. D.; CTGV. (1998). Doing with understanding: Lessons from research on problem- and project-based learning. Journal of the Learning Sciences, 7, 271–312.
   Web of Science ®Google Scholar
• Biswas, G., Jeong, H., Kinnebrew, J. S., Sulcer, B., & Roscoe, R. (2010). Measuring self-regulated learning skills through social interactions in a teachable agent environment. Research and Practice in Technology Enhanced Learning, 5(2), 123–152. doi:10.1142/S1793206810000839
   Google Scholar
• Bonawitz, E., Shafto, P., Gweon, H., Goodman, N. D., Spelke, E., & Schulz, L. (2011). The double-edged sword of pedagogy: Instruction limits spontaneous exploration and discovery. Cognition, 120(3), 322–330. doi:10.1016/j.cognition.2010.10.001
   PubMed Web of Science ®Google Scholar
• Bransford, J. D., & Schwartz, D. L. (1999). Rethinking transfer: A simple proposal with multiple implications. In A. Iran-Nejad & P. D. Pearson (Eds.), Review of research in education, 24 (pp. 61–101). Washington, DC: American Educational Research Association.
   Google Scholar
• Brown, A. L., Campione, J. C., & Day, J. D. (1981). Learning to learn: On training students to learn from texts. Educational Researcher, 10(2), 14–21. doi:10.3102/0013189X010002014
   Google Scholar
• Brown, T., & Wyatt, J. (2010). Design thinking for social innovation. Stanford Social Innovation Review, 8(1), 30–35.
   Google Scholar
• Carroll, M., Goldman, S., Britos, L., Koh, J., Royalty, A., & Hornstein, M. (2010). Destination, imagination and the fires within: Design thinking in a middle school classroom. International Journal of Art & Design Education, 29(1), 37–53. doi:10.1111/j.1476-8070.2010.01632.x
   Web of Science ®Google Scholar
• Chase, C., Chin, D. B., Oppezzo, M., & Schwartz, D. L. (2009). Teachable agents and the protégé effect: Increasing the effort towards learning. Journal of Science Education and Technology, 18, 334–352. doi:10.1007/s10956-009-9180-4
   Web of Science ®Google Scholar
• Chi, M., VanLehn, K., & Litman, D. (2010). Do micro-level tutorial decisions matter: Applying reinforcement learning to induce pedagogical tutorial tactics. In V. Aleven, J. Kay, & J. Mostow (Eds.), Intelligent tutoring systems. ITS 2010. Lecture notes in computer science, 6094 (pp. 224–234). Berlin, Heidelberg: Springer. doi:10.1007/978-3-642-13388-6_27
   Google Scholar
• Chi, M. T., Leeuw, N., Chiu, M. H., & LaVancher, C. (1994). Eliciting self‐explanations improves understanding. Cognitive Science, 18(3), 439–477.
   Web of Science ®Google Scholar
• Cohen, G. L., Garcia, J., Purdie-Vaughns, V., Apfel, N., & Brzustoski, P. (2009). Recursive processes in self-affirmation: Intervening to close the minority achievement gap. Science, 324(5925), 400–403. doi:10.1126/science.1170769
   PubMed Web of Science ®Google Scholar
• Conlin, L. D., Chin, D. B., Blair, K. P., Cutumisu, M., & Schwartz, D. L. (2015, June). Guardian angels of our better nature: Finding evidence of the benefits of design thinking. Proceedings of the 2015 Meeting of the American Society of Engineering Education, Seattle, WA. doi: 10.18260/p.24165
   Google Scholar
• Couzijn, M. (1999). Learning to write by observation of writing and reading processes; effects on learning and transfer. Learning and Instruction, 9(2), 109–142. doi:10.1016/S0959-4752(98)00040-1
   Web of Science ®Google Scholar
• Csikszentmihalyi, M., & Getzels, J. (1970). Concern for discovery: An attitudinal component of creative production 1. Journal of Personality, 38(1), 91–105.
   PubMed Web of Science ®Google Scholar
• Cutumisu, M., Blair, K. P., Chin, D. B., & Schwartz, D. L. (2015). Posterlet: A game-based assessment of children’s choices to seek feedback and revise. Journal of Learning Analytics, 2, 49–71. doi:10.18608/jla.2015.21.4
   Google Scholar
• Cutumisu, M., Blair, K. P., Chin, D. B., & Schwartz, D. L. (2016). Assessing whether students seek constructive criticism: The design of an automated feedback system for a graphic design task. International Journal of Artificial Intelligence in Education, 27(2). doi:10.1007/s40593-016-0137-5
   Google Scholar
• Cutumisu, M., & Schwartz, D. L. (2018). The impact of critical feedback choice on students’ revision, performance, learning, and memory. Computers in Human Behavior, 78, 351–367. doi:10.1016/j.chb.2017.06.029
   Web of Science ®Google Scholar
• Diamond, A., & Lee, K. (2011). Interventions shown to aid executive function development in children 4 to 12 years old. Science, 333(6045), 959–964. doi:10.1126/science.1204529
   PubMed Web of Science ®Google Scholar
• Dow, S. P., Glassco, A., Kass, J., Schwarz, M., Schwartz, D. L., & Klemmer, S. R. (2010). Parallel prototyping leads to better design results, more divergent creations, and self-efficacy gains. ACM Transactions on Computer-Human Interaction, 17, 4. doi:10.1145/1879831.1879836
   Google Scholar
• Dunbar, K., & Klahr, D. (1989). Developmental differences in scientific discovery processes. In D. Klahr & K. Kotovsky (Eds.), Complex information processing: The impact of Herbert A. Simon (pp. 109–143). New York, NY: Psychology Press.
   Google Scholar
• Dweck, C. (2006). Mindset: The new psychology of success. New York, NY: Random House.
   Google Scholar
• Fancsali, S. E. (2014, July). Causal discovery with models: Behavior, affect, and learning in cognitive tutor algebra. Proceedings of the 7th International Conference on Educational Data Mining, 28–35, London, UK.
   Google Scholar
• Feuerstein, R. (1979). The dynamic assessment of retarded performers: The learning potential assessment device, theory, instruments, and techniques. Baltimore, MD: University Park Press.
   Google Scholar
• Finkelstein, S. R., & Fishbach, A. (2012). Tell me what I did wrong: Experts seek and respond to negative feedback. Journal of Consumer Research, 39(1), 22–38. doi:10.1086/661934
   Web of Science ®Google Scholar
• Friedman, A. (Ed.) (2008). Framework for evaluating impacts of informal science education projects. Arlington, VA: National Science Foundation. Retrieved from http://caise.insci.org/uploads/docs/Eval_Framework.pdf
   Google Scholar
• Fuchs, L. S., Fuchs, D., Prentice, K., Burch, M., Hamlett, C. L., Owen, R., & Schroeter, K. (2003). Enhancing third-grade students’ mathematical problem solving with self-regulated learning strategies. Journal of Educational Psychology, 95(2), 306–315. doi:10.1037/0022-0663.95.2.306
   Web of Science ®Google Scholar
• Goldman, S., & Kabayadondo, Z. (Eds.). (2017). Taking design thinking to school: How the technology of design can transform teachers, learners, and classrooms. New York, NY: Routledge.
   Google Scholar
• Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. doi:10.3102/003465430298487
   Web of Science ®Google Scholar
• Hoskins, B., & Fredriksson, U. (2008). Learning to learn: What is it and can it be measured (Joint Research Centre Technical Report JRC, 46532).
   Google Scholar
• Ilgen, D. R., Fisher, C. D., & Taylor, M. S. (1979). Consequences of individual feedback on behavior in organizations. Journal of Applied Psychology, 64(4), 349. doi:10.1037/0021-9010.64.4.349
   Web of Science ®Google Scholar
• Iyengar, S. S., & Lepper, M. R. (1999). Rethinking the value of choice: A cultural perspective on intrinsic motivation. Journal of Personality and Social Psychology, 76(3), 349. doi:10.1037/0022-3514.76.3.349
   PubMed Web of Science ®Google Scholar
• Klehe, U. C., & Anderson, N. (2007). Working hard and working smart: Motivation and ability during typical and maximum performance. Journal of Applied Psychology, 92(4), 978. doi:10.1037/0021-9010.92.4.978
   PubMed Web of Science ®Google Scholar
• Kluger, A. N., & DeNisi, A. (1998). Feedback interventions: Toward the understanding of a double-edged sword. Current Directions in Psychological Science, 7(3), 67–72. doi:10.1111/1467-8721.ep10772989
   Web of Science ®Google Scholar
• Koedinger, K. R., Anderson, J. R., Hadley, W. H., & Mark, M. A. (1997). Intelligent tutoring goes to school in the big city. International Journal of Artificial Intelligence in Education, 8, 30–43.
   Google Scholar
• Koh, J. H. L., Chai, C. S., Wong, B., & Hong, H. Y. (2015). Design thinking for education: Conceptions and applications in teaching and learning. Singapore: Springer. doi:10.1007/978-981-287-444-3
   Google Scholar
• Kolodner, J. L., Crismond, D., Fasse, B. B., Gray, J., Holbrook, J., & Puntembakar, S. (2003). Putting a student-centered learning by designTM curriculum into practice: Lessons learned. Journal of the Learning Sciences, 12(4), 495–547. doi:10.1207/S15327809JLS1204_2
   Web of Science ®Google Scholar
• Kulik, J. A., & Kulik, C. C. (1988). Timing of feedback and verbal learning. Review of Educational Research, 58(1), 79–97. doi:10.3102/00346543058001079
   Web of Science ®Google Scholar
• Lee, S. J., Liu, Y. E., & Popović, Z. (2014, July). Learning individual behavior in an educational game: A data-driven approach. Proceedings of the 7th International Conference on Educational Data Mining, London, UK.
   Google Scholar
• Mory, E. H. (2003). Feedback research revisited. In D. H. Jonassen (Ed.), Handbook of research on educational communications and technology (pp. 745–783). Mahwah, NJ: Erlbaum.
   Google Scholar
• National Governors Association Center for Best Practices & Council of Chief State School Officers. (2010). Common core state standards. Washington, DC: Authors.
   Google Scholar
• National Research Council. (2009). Learning science in informal environments: People, places, and pursuits. Washington, DC: The National Academies Press. doi:10.17226/12190
   Google Scholar
• NGSS Lead States. (2013). Next generation science standards: For States, By States. Washington, DC: The National Academies Press.
   Google Scholar
• Noweski, C., Scheer, A., Büttner, N., von Thienen, J., Erdmann, J., & Meinel, C. (2012). Towards a paradigm shift in education practice: Developing twenty-first century skills with design thinking. In H. Plattner, C. Meinel, & L. Leifer (Eds.), Design thinking research (pp. 71–94). Berlin, Germany: Springer. doi:10.1007/978-3-642-31991-4_5
   Google Scholar
• Oppezzo, M. A., & Schwartz, D. L. (2013). A behavior change perspective on self-regulated learning with teachable agents. In R. Azevedo & V. Alevan (Eds.), International handbook of metacognition and learning (pp. 485–500). New York, NY: Springer. doi:10.1007/978-1-4419-5546-3_31
   Google Scholar
• Osborne, J., Erduran, S., & Simon, S. (2004). Enhancing the quality of argumentation in school science. Journal of Research in Science Teaching, 41(10), 994–1020. doi:10.1002/(ISSN)1098-2736
   Web of Science ®Google Scholar
• Peacock, S. B., Smith, C., Martin, T., Aghababyan, A., Popović, Z., Andersen, E., & Liu, Y. E. (2013). Learning fractions through splitting in an online game. Paper presented at the 2013 Annual Meeting of the American Educational Research Association, San Francisco, CA.
   Google Scholar
• Pintrich, P. R., & De Groot, E. V. (1990). Motivational and self-regulated learning components of classroom academic performance. Journal of Educational Psychology, 82(1), 33–40. doi:10.1037/0022-0663.82.1.33
   Web of Science ®Google Scholar
• Razzouk, R., & Shute, V. (2012). What is design thinking and why is it important? Review of Educational Research, 82(3), 330–348. doi:10.3102/0034654312457429
   Web of Science ®Google Scholar
• Ridley, D. S., Schutz, P. A., Glanz, R. S., & Weinstein, C. E. (1992). Self-regulated learning: The interactive influence of metacognitive awareness and goal-setting. Journal of Experimental Education, 60(4), 293–306. doi:10.1080/00220973.1992.9943867
   Web of Science ®Google Scholar
• Roll, I., Aleven, V., McLaren, B. M., & Koedinger, K. R. (2011). Improving students’ help-seeking skills using metacognitive feedback in an intelligent tutoring system. Learning and Instruction, 21(2), 267–280. doi:10.1016/j.learninstruc.2010.07.004
   Web of Science ®Google Scholar
• Roll, I., Baker, R. S., Aleven, V., & Koedinger, K. R. (2014). The effect of overuse and underuse of help resources in intelligent tutoring systems. Journal of the Learning Sciences. doi:10.1080/10508406.2014.883977
   Google Scholar
• Rotherham, A. J., & Willingham, D. (2009). 21st century. Educational Leadership, 67(1), 16–21.
   Web of Science ®Google Scholar
• Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55(1), 68–78.
   PubMed Web of Science ®Google Scholar
• Sackett, P. R., Zedeck, S., & Fogli, L. (1988). Relations between measures of typical and maximum job performance. Journal of Applied Psychology, 73(3), 482. doi:10.1037/0021-9010.73.3.482
   Web of Science ®Google Scholar
• Sadler, P. M., Coyle, H. P., & Schwartz, M. (2000). Engineering competitions in the middle school classroom: Key elements in developing effective design challenges. Journal of the Learning Sciences, 9(3), 299–327. doi:10.1207/S15327809JLS0903_3
   Web of Science ®Google Scholar
• Schwartz, D. L., & Arena, D. (2013). Measuring what matters most: Choice-based assessments for the digital age. Cambridge, MA: MIT Press. Retrieved from https://mitpress.mit.edu/books/measuring-what-matters-most
   Google Scholar
• Schwartz, D. L., Cheng, K. M., Salehi, S., & Wieman, C. (2016). Commentary: The half-empty question for socio-cognitive interventions. Journal of Educational Psychology, 108(3), 397–404. doi:10.1037/edu0000122
   Web of Science ®Google Scholar
• Schwartz, D. L., & Martin, T. (2004). Inventing to prepare for learning: The hidden efficiency of original student production in statistics instruction. Cognition & Instruction, 22, 129–184. doi:10.1207/s1532690xci2202_1
   Web of Science ®Google Scholar
• Shute, V., & Ventura, M. (2013). Stealth assessment: Measuring and supporting learning in video games. Cambridge, MA: MIT Press. Retrieved from https://mitpress.mit.edu/books/stealth-assessment
   Google Scholar
• Shute, V. J., Ventura, M., Bauer, M. I., & Zapata-Rivera, D. (2009). Melding the power of serious games and embedded assessment to monitor and foster learning: Flow and grow. In U. Ritterfeld, M. Cody, & P. Vorderer (Eds.), Serious games: Mechanisms and effects (pp. 295–321). New York, NY: Routledge.
   Google Scholar
• Silk, E. M., Higashi, R., Shoop, R., & Schunn, C. D. (2010). Designing technology activities that teach mathematics. The Technology Teacher, 69(4), 21–27.
   Google Scholar
• Snow, E. L., Allen, L. K., Russell, D. G., & McNamara, D. S. (2014). Who’s in control?: Categorizing nuanced patterns of behaviors within a game-based intelligent tutoring system. In J. Stamper, Z. Pardos, M. Mavrikis, & B. M. McLaren (Eds.), Proceedings of the 7th international conference on educational data mining (pp. 185–192). London, UK: Springer.
   Google Scholar
• Spörer, N., Brunstein, J. C., & Kieschke, U. (2009). Improving students’ reading comprehension skills: Effects of strategy instruction and reciprocal teaching. Learning and Instruction, 19(3), 272–286. doi:10.1016/j.learninstruc.2008.05.003
   Web of Science ®Google Scholar
• Thorndike, E. L. (1927). The law of effect. American Journal of Psychology, 39, 212–222. doi:10.2307/1415413
   Web of Science ®Google Scholar
• Trilling, B., & Fadel, C. (2009). 21st century skills: Learning for life in our times. New York, NY: John Wiley & Sons.
   Google Scholar
• Ventura, M., & Shute, V. (2013). The validity of a game-based assessment of persistence. Computers in Human Behavior, 29(6), 2568–2572. doi:10.1016/j.chb.2013.06.033
   Web of Science ®Google Scholar
• Wirth, K. R., & Perkins, D. (2008). Learning to learn. Retrieved from: http://www.macalester.edu/geology/wirth/CourseMaterials.html
   Google Scholar
• Zimmerman, B. Z. (2002). Becoming a self-regulated learner: An overview. Theory into Practice, 41(2), 64–70. doi:10.1207/s15430421tip4102_2
   Web of Science ®Google Scholar