Evaluating instructional designs with mental workload assessments in university classrooms

References

• Ayres, Paul. 2006. “Using Subjective Measures to Detect Variations of Intrinsic Cognitive Load Within Problems.” Learning and Instruction 16 (5): 389–400.
   Web of Science ®Google Scholar
• Baddeley, Alan, and Graham James Hitch. 1974. “Working Memory.” In Recent Advances in Learning and Motivation, Vol. 8, edited by G.A. Bower, 47–90. Academic Press.
   Google Scholar
• Box, Joan Fisher. 1987. “Guinness, Gosset, Fisher, and Small Samples.” Statistical Science 2 (1): 45–52.
   Google Scholar
• Brame, Cynthia J. 2016. “Effective Educational Videos: Principles and Guidelines for Maximizing Student Learning from Video Content.” CBE-Life Sciences Education 15 (4): es6.
   PubMed Web of Science ®Google Scholar
• Brookhuis, Karel A., and Dick de Waard. 2010. “Monitoring Drivers Ḿental Workload in Driving Simulators Using Physiological Measures.” Accident Analysis & Prevention 42 (3): 898–903.
   PubMed Web of Science ®Google Scholar
• Brünken, Roland Ed., Jan L. Plass, and Roxana Ed. Moreno. 2010. ““Current Issues and Open Questions in Cognitive Load Research.”
   Google Scholar
• Cain, Brad. 2007. A Review of the Mental Workload Literature, Technical report, Defence Research and Development Canada Toronto, Human System Integration Section.
   Google Scholar
• Chandler, P., and J. Sweller. 1991. “Cognitive Load Theory and the Format of Instruction.” Cognition and Instruction 8 (4): 293–332.
   Web of Science ®Google Scholar
• Cierniak, Gabriele, Katharina Scheiter, and Peter Gerjets. 2009. “Explaining the Split-attention Effect: Is the Reduction of Extraneous Cognitive Load Accompanied by An Increase in Germane Cognitive Load?.” Computers in Human Behavior 25 (2): 315–324.
   Web of Science ®Google Scholar
• Clark, Richard E., and David F. Feldon. 2005. “Five Common But Questionable Principles of Multimedia Learning.” The Cambridge Handbook of Multimedia Learning 6: 97–116.
   Google Scholar
• Debue, Nicolas, and Cécile van de Leemput. 2014. “What Does Germane Load Mean? An Empirical Contribution to the Cognitive Load Theory.” Frontiers in Psychology 5: 1099.
   PubMed Web of Science ®Google Scholar
• De Jong, Ton. 2010. “Cognitive Load Theory, Educational Research, and Instructional Design: Some Food for Thought.” Instructional Science 38 (2): 105–134.
   Web of Science ®Google Scholar
• DeLeeuw, Krista E., and Richard E. Mayer. 2008. “A Comparison of Three Measures of Cognitive Load: Evidence for Separable Measures of Intrinsic, Extraneous, and Germane Load.” Journal of Educational Psychology 100 (1): 223.
   Web of Science ®Google Scholar
• DiDomenico, Angela, and Maury A. Nussbaum. 2008. “Interactive Effects of Physical and Mental Workload on Subjective Workload Assessment.” International Journal of Industrial Ergonomics 38 (11–12): 977–983.
   Web of Science ®Google Scholar
• Dixon, Peter. 1991. “From Research to Theory to Practice: Commentary on Chandler and Sweller.” Cognition and Instruction 8 (4): 343–350.
   Web of Science ®Google Scholar
• Doksum, Kjell. 1974. “Empirical Probability Plots and Statistical Inference for Nonlinear Models in the Two-sample Case.” The Annals of Statistics 00: 267–277.
   Web of Science ®Google Scholar
• Gerjets, Peter, Katharina Scheiter, and Gabriele Cierniak. 2009. “The Scientific Value of Cognitive Load Theory: A Research Agenda Based on the Structuralist View of Theories.” Educational Psychology Review 21 (1): 43–54.
   Web of Science ®Google Scholar
• Goldman, Susan R. 1991. “On the Derivation of Instructional Applications From Cognitive Theories: Commentary on Chandler and Sweller.” Cognition and Instruction 8 (4): 333–342.
   Web of Science ®Google Scholar
• Gwizdka, Jacek. 2009. “Assessing Cognitive Load on Web Search Tasks.” The Ergonomic Open Journal2 (1): 114–123.
   Google Scholar
• Gwizdka, Jacek. 2010, November. “Distribution of Cognitive Load in Web Search.” Journal of the American Society & Information Science & Technology 61 (11): 2167–2187.
   Web of Science ®Google Scholar
• Harrell, Frank E., and C. E. Davis. 1982. “A New Distribution-free Quantile Estimator.” Biometrika 69 (3): 635–640.
   Web of Science ®Google Scholar
• Hart, Sandra G. 2006a. “Nasa-Task Load Index (nasa-tlx); 20 Years Later.” In Human Factors and Ergonomics Society Annual Meeting, Vol. 50, 904–908. San Francisco, CA: Sage Journals.
   Google Scholar
• Hart, Sandra G. 2006b. “Nasa-task Load Index (nasa-tlx); 20 Years Later.” Proceedings of the Human Factors and Ergonomics Society Annual Meeting 50 (9): 904–908.
   Google Scholar
• Hart, Sandra G., and Lowell E. Staveland. 1988. “Development of Nasa-Tlx (Task Load Index): Results of Empirical and Theoretical Research.” In Human Mental Workload, edited by Peter A. Hancock and Najmedin Meshkati, Vol. 52 of Advances in Psychology, 139–183. North-Holland.
   Google Scholar
• Henrie, Curtis R., Lisa R. Halverson, and Charles R. Graham. 2015. “Measuring Student Engagement in Technology-mediated Learning: A Review.” Computers & Education 90: 36–53.
   Web of Science ®Google Scholar
• Homer, B. D., J. L. Plass, and L. Blake. 2008. “The Effects of Video on Cognitive Load and Social Presence in Multimedia-learning.” Computers in Human Behavior 24 (3): 786–797.
   Web of Science ®Google Scholar
• Huey, Beverly Messick, and Christopher D. Wickens. 1993. Workload Transition: Implication for Individual and Team Performance. Washington, DC: National Academy Press.
   Google Scholar
• Jiang, Dayu, and Slava Kalyuga. 2020. “Confirmatory Factor Analysis of Cognitive Load Ratings Supports a Two-factor Model.” Tutorials in Quantitative Methods for Psychology 16: 216–225.
   Web of Science ®Google Scholar
• Kalyuga, Slava. 2007. “Enhancing Instructional Efficiency of Interactive E-learning Environments: A Cognitive Load Perspective.” Educational Psychology Review 19 (3): 387–399.
   Web of Science ®Google Scholar
• Kalyuga, Slava. 2011. “Cognitive Load Theory: How Many Types of Load Does it Really Need?.” Educational Psychology Review 23 (1): 1–19.
   Web of Science ®Google Scholar
• Kalyuga, Slava, and Anne-Marie Singh. 2016. “Rethinking the Boundaries of Cognitive Load Theory in Complex Learning.” Educational Psychology Review 28 (4): 831–852.
   Web of Science ®Google Scholar
• Levinson, A. J. 2010. “Where is Evidence-based Instructional Design in Medical Education Curriculum Development?.” Medical Education 44: 536–537.
   PubMed Web of Science ®Google Scholar
• Longo, Luca. 2014. Formalising Human Mental Workload as a Defeasible Computational Concept, Doctor in philosophy, School of Computer Science and Statistics -- Trinity College Dublin.
   Google Scholar
• Longo, Luca, and Pierpaolo Dondio. 2015. “On the Relationship Between Perception of Usability and Subjective Mental Workload of Web Interfaces.” In Web Intelligence and Intelligent Agent Technology (WI-IAT), 2015 IEEE/WIC/ACM International Conference on, Vol. 1, 345–352. IEEE.
   Google Scholar
• Mautone, P. D., and R. E Mayer. 2001. “Signaling As a Cognitive Guide in Multimedia Learning.” Journal of Educational Psychology 93 (2): 377.
   Web of Science ®Google Scholar
• Mayer, Richard E. 1997. “Multimedia Learning: Are We Asking the Right Questions?.” Educational Psychologist 32 (1): 1–19.
   Web of Science ®Google Scholar
• Mayer, Richard E. 2002. “Multimedia Learning.” Psychology of Learning and Motivation 41: 85–139.
   Web of Science ®Google Scholar
• Mayer, Richard E. 2005. The Cambridge Handbook of Multimedia Learning. Cambridge: Cambridge University Press.
   Google Scholar
• Mayer, Richard E. 2009. Multimedia Learning. Cambridge: Cambridge University Press.
   Google Scholar
• Mayer, Richard E. 2012. “Information Processing.” 85–99. American Psychological Association.
   Google Scholar
• Mayer, R. E. 2017. “Using Multimedia for E-learning.” Journal of Computer Assisted Learning 33 (5): 403–423. JCAL-16-266.R1.
   Web of Science ®Google Scholar
• Mayer, Richard E., and Roxana Moreno. 2003. “Nine Ways to Reduce Cognitive Load in Multimedia Learning.” Educational Psychologist 38 (1): 43–52.
   Web of Science ®Google Scholar
• Miller, G. A. 1956. “The Magical Number Seven, Plus Or Minus Two: Some Limits on Our Capacity for Processing Information.” Psychological Review 63 (2): 81–97.
   PubMed Web of Science ®Google Scholar
• Moroney, Brian W., Joel S. Warm, and William N. Dember. 1995. “Effects of Demand Transitions on Vigilance Performance and Perceived Workload.” Proceedings of the Human Factors and Ergonomics Society Annual Meeting 39 (21): 1375–1379.
   Google Scholar
• Mousavi, S., R. Low, and J. Sweller. 1995. “Reducing Cognitive Load by Mixing Auditory and Visual Presentation Modes.” Journal of Educational Psychology 87 (2): 319–334.
   Web of Science ®Google Scholar
• Moustafa, Karim, and Luca Longo. 2019. “Analysing the Impact of Machine Learning to Model Subjective Mental Workload: A Case Study in Third-Level Education.” In Human Mental Workload: Models and Applications, edited by Luca Longo and M. Chiara Leva, 92–111, Cham: Springer International Publishing.
   Google Scholar
• Moustafa, Karim, Saturnino Luz, and Luca Longo. 2017. “Assessment of Mental Workload: A Comparison of Machine Learning Methods and Subjective Assessment Techniques.” In International Symposium on Human Mental Workload: Models and Applications, 30–50. Springer.
   Google Scholar
• Noyes, Jan M, and Daniel P. J. Bruneau. 2007. “A Self-analysis of the Nasa-tlx Workload Measure.” Ergonomics 50 (4): 514–519.
   PubMed Web of Science ®Google Scholar
• Orru, Giuliano, Federico Gobbo, Declan O'Sullivan, and Luca Longo. 2018. “An Investigation of the Impact of a Social Constructivist Teaching Approach, Based on Trigger Questions, Through Measures of Mental Workload and Efficiency.” In Proceedings of the 10th International Conference on Computer Supported Education, CSEDU 2018, Funchal, Madeira, Portugal, March 15-17, 2018, Volume 2., 292–302.
   Google Scholar
• Orru, Giuliano, and Luca Longo. 2019a. “Direct Instruction and its Extension With a Community of Inquiry: A Comparison of Mental Workload, Performance and Efficiency.” In Proceedings of the 11th International Conference on Computer Supported Education, CSEDU 2019, Heraklion, Crete, Greece, May 2–4, 2019, Volume 1., 436–444.
   Google Scholar
• Orru, Giuliano, and Luca Longo. 2019b. “The Evolution of Cognitive Load Theory and the Measurement of its Intrinsic, Extraneous and Germane Loads: A Review.” In Human Mental Workload: Models and Applications, edited by Luca Longo and M. Chiara Leva, 23–48. Cham: Springer International Publishing.
   Google Scholar
• Paas, Fred, Juhani E. Tuovinen, Huib Tabbers, and Pascal W. M. Van Gerven. 2003. “Cognitive Load Measurement As a Means to Advance Cognitive Load Theory.” Educational Psychologist 38 (1): 63–71.
   Web of Science ®Google Scholar
• Paas, Fred, and Jeroen J. G. Van Merrienboer. 1993. “The Efficiency of Instructional Conditions: An Approach to Combine Mental Effort and Performance Measures.” Human Factors: the Journal of the Human Factors and Ergonomics Society 35 (4): 737–743.
   Web of Science ®Google Scholar
• Paivio, A. 1990. Mental Representations: A Dual Coding Approach. Oxford Psychology Series. Oxford University Press.
   Google Scholar
• Popper, Karl. 2005. The Logic of Scientific Discovery. London: Routledge.
   Google Scholar
• Popper, Karl. 2014. Conjectures and Refutations: The Growth of Scientific Knowledge. New York: Routledge.
   Google Scholar
• Puma, Sébastien, Nadine Matton, Pierre-Vincent Paubel, and André Tricot. 2018. “Cognitive Load Theory and Time Considerations: Using the Time-based Resource Sharing Model.” Educational Psychology Review 30 (3): 1199–1214.
   Web of Science ®Google Scholar
• Reid, Gary B., and Thomas E. Nygren. 1988. “The Subjective Workload Assessment Technique: A Scaling Procedure for Measuring Mental Workload.” In Human Mental Workload, Vol. 52 of Advances in Psychology, Chap. 8, edited by Peter A. Hancock and Najmedin Meshkati, 185–218. North-Holland.
   Google Scholar
• Rizzo, Lucas, Pierpaolo Dondio, Sarah Jane Delany, and Luca Longo. 2016. “Modeling Mental Workload Via Rule-Based Expert System: A Comparison With Nasa-Tlx and Workload Profile.” In IFIP International Conference on Artificial Intelligence Applications and Innovations, 215–229. Springer.
   Google Scholar
• Rizzo, Lucas Middeldorf, and Luca Longo. 2017. “Representing and Inferring Mental Workload Via Defeasible Reasoning: A Comparison With the Nasa Task Load Index and the Workload Profile.” In Proceedings of the 1st Workshop on Advances In Argumentation In Artificial Intelligence AI3@AI*IA. CEURS.
   Google Scholar
• Rizzo, Lucas, and Luca Longo. 2018. “Inferential Models of Mental Workload With Defeasible Argumentation and Non-Monotonic Fuzzy Reasoning: A Comparative Study.” In Proceedings of the 2nd Workshop on Advances In Argumentation In Artificial Intelligence, co-located with XVII International Conference of the Italian Association for Artificial Intelligence, AI3@AI*IA 2018, 20-23 November 2018, Trento, Italy, 11–26. CEURS.
   Google Scholar
• Roscoe, Alan H., and George A. Ellis. 1990, March. A Subjective Rating Scale for Assessing Pilot Workload in Flight: A Decade of Practical Use, Technical report TR 90019, Royal Aerospace Establishment, Farnborough (UK).
   Google Scholar
• Rousselet, Guillaume A., Cyril R. Pernet, and Rand R. Wilcox. 2017. “Beyond Differences in Means: Robust Graphical Methods to Compare Two Groups in Neuroscience.” European Journal of Neuroscience 46 (2): 1738–1748.
   PubMed Web of Science ®Google Scholar
• Royston, Patrick. 1995. “Remark AS R94: A Remark on Algorithm AS 181: The W-test for Normality.” Journal of the Royal Statistical Society. Series C (Applied Statistics) 44 (4): 547–551.
   Web of Science ®Google Scholar
• Rubio, Susana, Eva Diaz, Jesus Martin, and Jose M. Puente. 2004. “Evaluation of Subjective Mental Workload: A Comparison of Swat, Nasa-tlx, and Workload Profile Methods.” Applied Psychology 53 (1): 61–86.
   Web of Science ®Google Scholar
• Schnotz, Wolfgang, and Christian Kürschner. 2007. “A Reconsideration of Cognitive Load Theory.” Educational Psychology Review 19 (4): 469–508.
   Web of Science ®Google Scholar
• Seufert, Tina, Inge Jänen, and Roland Brünken. 2007. “The Impact of Intrinsic Cognitive Load on the Effectiveness of Graphical Help for Coherence Formation.” Computers in Human Behavior 23 (3): 1055–1071.
   Web of Science ®Google Scholar
• Sweller, John. 1994. “Cognitive Load Theory, Learning Difficulty, and Instructional Design.” Learning and Instruction 4 (4): 295–312.
   Google Scholar
• Sweller, John. 2010. “Element Interactivity and Intrinsic, Extraneous, and Germane Cognitive Load.” Educational Psychology Review 22 (2): 123–138.
   Web of Science ®Google Scholar
• Sweller, J., J. Van Merrienboer, and F. Paas. 1998. “Cognitive Architecture and Instructional Design.” Educational Psychology Review 10 (3): 251–296.
   Web of Science ®Google Scholar
• Tattersall, Andrew J., and Penelope S. Foord. 1996. “An Experimental Evaluation of Instantaneous Self-assessment As a Measure of Workload.” Ergonomics 39 (5): 740–748.
   PubMed Web of Science ®Google Scholar
• Tsang, Pamela S. 2006. “Mental Workload.” In Encyclopedia of Ergonomics and Human Factors, Vol. 1, Chap. 166. edited by Waldemar Karwowski, Taylor & Francis.
   Google Scholar
• Tsang, Pamela S., and Velma L. Velazquez. 1996. “Diagnosticity and Multidimensional Subjective Workload Ratings.” Ergonomics 39 (3): 358–381.
   PubMed Web of Science ®Google Scholar
• Tsang, Pamela S., and Michael A. Vidulich. 2006. “Mental Workload and Situation Awareness.” In edited by Gavriel Salvendy, Handbook of Human Factors and Ergonomics, 243–268. Wiley & Sons.
   Google Scholar
• Van Gog, Tamara, Femke Kirschner, Liesbeth Kester, and Fred Paas. 2012. “Timing and Frequency of Mental Effort Measurement: Evidence in Favour of Repeated Measures.” Applied Cognitive Psychology26 (6): 833–839.
   Web of Science ®Google Scholar
• Van Gog, Tamara, and Fred Paas. 2008. “Instructional Efficiency: Revisiting the Original Construct in Educational Research.” Educational Psychologist 43 (1): 16–26.
   Web of Science ®Google Scholar
• Vidulich, Michael A., James Schueren, and Frederic G. Ward. 1991, December. “Using the Subjective Workload Dominance (sword) Technique for Projective Workload Assessment.” Human Factors Society33 (6): 677–691.
   Web of Science ®Google Scholar
• Wickens, Christopher D. 2008. “Multiple Resources and Mental Workload.” Human Factors 50 (2): 449–454.
   PubMed Web of Science ®Google Scholar
• Wickens, Christopher D., and Justin G. Hollands. 1999, September. Engineering Psychology and Human Performance. 3rd ed. New York: Prentice Hall.
   Google Scholar
• Wiebe, Eric N., Edward Roberts, and Tara S. Behrend. 2010. “An Examination of Two Mental Workload Measurement Approaches to Understanding Multimedia Learning.” Computers in Human Behavior 26 (3): 474–481.
   Web of Science ®Google Scholar
• Wilcox, Rand R. 1995. “Comparing Two Independent Groups Via Multiple Quantiles.” Journal of the Royal Statistical Society: Series D (The Statistician) 44 (1): 91–99.
   Google Scholar
• Wilcox, Rand R. 2012. “Comparing Two Independent Groups Via a Quantile Generalization of the Wilcoxon-mann-whitney Test.” Journal of Modern Applied Statistical Methods 11 (2): 2.
   Google Scholar
• Wilcox, Rand R., and David M. Erceg-Hurn. 2012. “Comparing Two Dependent Groups Via Quantiles.” Journal of Applied Statistics 39 (12): 2655–2664.
   Web of Science ®Google Scholar
• Wilcox, Rand R., David M. Erceg-Hurn, Florence Clark, and Michael Carlson. 2014. “Comparing Two Independent Groups Via the Lower and Upper Quantiles.” Journal of Statistical Computation and Simulation 84 (7): 1543–1551.
   Web of Science ®Google Scholar
• Wilcoxon, Frank. 1945, December. “Individual Comparisons by Ranking Methods.” Biometrics Bulletin1 (6): 80–83.
   Google Scholar
• Wilson, Glenn F., and Thomas F. Eggemeier. 2006. “Mental Workload Measurement.” In Int. Encyclopedia of Ergonomics and Human Factors, 2nd ed., Vol. 1, Chap. 167. edited by Waldemar Karwowski, Taylor and Francis.
   Google Scholar
• Wilson, Glenn F., and Christopher A. Russell. 2003. “Real-time Assessment of Mental Workload Using Psychophysiological Measures and Artificial Neural Networks.” Human Factors 45 (4): 635–644.
   PubMed Web of Science ®Google Scholar
• Xie, Bin, and Gavriel Salvendy. 2000. “Review and Reappraisal of Modelling and Predicting Mental Workload in Single and Multi-task Environments.” Work and Stress 14 (1): 74–99.
   Web of Science ®Google Scholar
• Young, Mark S., Karel A. Brookhuis, Christopher D. Wickens, and Peter A. Hancock. 2015. “State of Science: Mental Workload in Ergonomics.” Ergonomics 58 (1): 1–17.
   PubMed Web of Science ®Google Scholar
• Young, M. S., and N. A. Stanton. 2001. “Mental Workload: Theory, Measurement, and Application.” In International Encyclopedia of Ergonomics and Human Factors, Vol. 1, 507–509. London: Taylor and Francis.
   Google Scholar
• Young, Mark S., and Neville A. Stanton. 2004. “Mental Workload.” In Handbook of Human Factors and Ergonomics Methods, Chap. 39, edited by Neville Anthony Stanton, Alan Hedge, Karel Brookhuis, Eduardo Salas, and Hal W. Hendrick, 1–9. CRC Press.
   Google Scholar
• Young, Mark S., and Neville A. Stanton. 2006. “Mental Workload: Theory, Measurement, and Application.” In Encyclopedia of Ergonomics and Human Factors, 2nd ed., Vol. 1, edited by Waldermar Karwowski, 818–821. Taylor & Francis.
   Google Scholar
• Zheng, Robert Z. 2017. Cognitive Load Measurement and Application: a Theoretical Framework for Meaningful Research and Practice. New York: Routledge.
   Google Scholar
• Zijlstra, Ferdinand R. H. 1993. “Efficiency in Work Behaviour.” Doctoral thesis, Delft University.
   Google Scholar