Thinking Aloud in the Presence of Interruptions and Time Constraints

REFERENCES

• Bailey , B. P. and Iqbal , S. T. 2008 . Understanding changes in mental workload during execution of goal-directed tasks and its application for interruption management . ACM Transactions on Computer-Human Interaction , 14 : 21:01 – 21:28 .
   Web of Science ®Google Scholar
• Beatty , J. 1982 . Task-evoked pupillary responses, processing load, and the structure of processing resources . Psychological Bulletin , 91 : 276 – 292 .
   PubMed Web of Science ®Google Scholar
• Bernhardt , P. C. , Dabbs , J. M. and Riad , J. K. 1996 . Pupillometry system for use in social psychology . Behavior Research Methods, Instrumentation, & Computers , 28 : 61 – 66 .
   Google Scholar
• Boren , T. and Ramey , J. 2000 . Thinking aloud: Reconciling theory and practice . IEEE Transactions on Professional Communication , 43 : 261 – 278 .
   Web of Science ®Google Scholar
• Chi , M. T. H. , De Leeuw , N. , Chiu , M.-H. and LaVancher , C. 1994 . Eliciting self-explanations improves understanding . Cognitive Science , 18 : 439 – 477 .
   Web of Science ®Google Scholar
• Cooke , L. 2010 . Assessing concurrent think-aloud protocol as a usability test method: A technical communication approach . IEEE Transactions on Professional Communication , 53 : 202 – 215 .
   Web of Science ®Google Scholar
• Dickson , J. , McLennan , J. and Omodei , M. M. 2000 . Effects of concurrent verbalization on a time-critical, dynamic decision-making task . Journal of General Psychology , 127 : 217 – 228 .
   PubMed Web of Science ®Google Scholar
• Dumas , J. S. and Fox , J. E. 2008 . “ Usability testing: Current practice and future directions ” . In The human–computer interaction handbook: Fundamentals, evolving technologies, and emerging applications , 2nd , Edited by: Sears , A. and Jacko , J. A. 1129 – 1149 . New York , NY : Erlbaum .
   Google Scholar
• Dumas , J. S. and Redish , J. C. 1999 . A practical guide to usability testing. , Revised , Exeter , , UK : Intellect Books .
   Google Scholar
• Emerson , M. J. and Miyake , A. 2003 . The role of inner speech in task switching: A dual-task investigation . Journal of Memory and Language , 48 : 148 – 168 .
   Web of Science ®Google Scholar
• Ericsson , K. A. and Simon , H. A. 1980 . Verbal reports as data . Psychological Review , 87 : 215 – 251 .
   Web of Science ®Google Scholar
• Ericsson , K. A. and Simon , H. A. 1993 . Protocol analysis: Verbal reports as data. , Revised , Cambridge , MA : MIT Press .
   Google Scholar
• Fleiss , J. 1981 . Statistical methods for rates and proportions , New York , NY : Wiley .
   Google Scholar
• Fox , M. C. , Ericsson , K. A. and Best , R. 2011 . Do procedures for verbal reporting of thinking have to be reactive? A meta-analysis and recommendations for best reporting methods . Psychological Bulletin , 137 : 316 – 344 .
   PubMed Web of Science ®Google Scholar
• Gilhooly , K. J. , Fioratou , E. and Henretty , N. 2010 . Verbalization and problem solving: Insight and spatial factors . British Journal of Psychology , 101 : 81 – 93 .
   PubMed Web of Science ®Google Scholar
• Goldberg , J. H. and Kotval , X. P. 1999 . Computer interface evaluation using eye movements: Methods and constructs . International Journal of Industrial Ergonomics , 24 : 631 – 645 .
   Web of Science ®Google Scholar
• González , V. M. and Mark , G. 2004 . “ Constant, constant, multi-tasking craziness”: Managing multiple working spheres ” . In Proceedings of the CHI 2004 Conference on Human Factors in Computing Systems , 113 – 120 . New York : ACM Press .
   Google Scholar
• Gulliksen , J. , Boivie , I. , Persson , J. , Hektor , A. and Herulf , L. 2004 . “ Making a difference—A survey of the usability profession in Sweden ” . In Proceedings of the NordiCHI 2004 Conference on Human–Computer Interaction , 207 – 215 . New York , NY : ACM Press .
   Google Scholar
• Haak , M. J. v. d. , Jong , M. D. T. d. and Schellens , P. J. 2003 . Retrospective vs. concurrent think-aloud protocols: Testing the usability of an online library catalogue . Behaviour & Information Technology , 22 : 339 – 351 .
   Web of Science ®Google Scholar
• Haak , M. J. v. d. , Jong , M. D. T. d. and Schellens , P. J. 2007 . Evaluation of an informational web site: Three variants of the think-aloud method compared . Technical Communication , 54 : 58 – 71 .
   Web of Science ®Google Scholar
• Hart , S. G. and Staveland , L. E. 1988 . “ Development of NASA–TLX (task load index): Results of empirical and theoretical research ” . In Human mental workload , Edited by: Hancock , P. A. and Meshkati , N. 139 – 183 . Amsterdam, the , , Netherlands : North-Holland .
   Google Scholar
• Held , J. E. and Biers , D. W. 1992 . “ Software usability testing: Do evaluator intervention and task structure make any difference? ” . In Proceedings of the Human Factors Society 36th Annual Meeting 1215 – 1219 . Santa Monica , CA: HFS
   Google Scholar
• Hendy , K. , Hamilton , K. M. and Landry , L. N. 1993 . Measuring subjective workload: When is one scale better than many? . Human Factors , 35 : 579 – 601 .
   Web of Science ®Google Scholar
• Hertzum , M. , Hansen , K. D. and Andersen , H. H. K. 2009 . Scrutinising usability evaluation: Does thinking aloud affect behaviour and mental workload? . Behaviour & Information Technology , 28 : 165 – 181 .
   Web of Science ®Google Scholar
• Hertzum , M. and Holmegaard , K. D. 2013 . Perceived time as a measure of mental workload: Effects of time constraints and task success . International Journal of Human–Computer Interaction , 29 : 26 – 39 .
   Web of Science ®Google Scholar
• James , W. 1890 . The principles of psychology , New York , NY : Dover .
   Google Scholar
• Karbach , J. and Kray , J. 2007 . Developmental changes in switching between mental task sets: The influence of verbal labeling in childhood . Journal of Cognition and Development , 8 : 205 – 236 .
   Web of Science ®Google Scholar
• Kokini , C. M. , Lee , S. , Koubek , R. J. and Moon , S. K. 2012 . Considering context: The role of mental workload and operator control in users' perceptions of usability . International Journal of Human–Computer Interaction , 28 : 543 – 559 .
   Web of Science ®Google Scholar
• Lewis , C. 1982 . Using the “thinking-aloud” method in cognitive interface design (RC 9265, #40713) , Yorktown Heights , NY : IBM Thomas Watson Research Center .
   Google Scholar
• MacLeod , C. M. 1991 . Half a century of research on the Stroop effect: An integrative review . Psychological Bulletin , 109 : 163 – 203 .
   PubMed Web of Science ®Google Scholar
• McFarlane , D. C. 2002 . Comparison of four primary methods for coordinating the interruption of people in human–computer interaction . Human–Computer Interaction , 17 : 63 – 139 .
   Web of Science ®Google Scholar
• Meissner , C. A. and Brigham , J. C. 2001 . A meta-analysis of the verbal overshadowing effect in face identification . Applied Cognitive Psychology , 15 : 603 – 616 .
   Web of Science ®Google Scholar
• Mintzberg , H. 1975 . The manager's job: Folklore and fact . Harvard Business Review , 53 ( 4 ) : 49 – 61 .
   Google Scholar
• Monsell , S. 2003 . Task switching . Trends in Cognitive Sciences , 7 : 134 – 140 .
   PubMed Web of Science ®Google Scholar
• Nielsen , J. 1993 . Usability engineering , Boston , MA : Academic Press .
   Google Scholar
• Nisbett , R. E. and Wilson , T. D. 1977 . Telling more than we can know: Verbal reports on mental processes . Psychological Review , 84 : 231 – 259 .
   Web of Science ®Google Scholar
• Nygren , T. E. 1991 . Psychometric properties of subjective workload measurement techniques: Implications for their use in the assessment of perceived mental workload . Human Factors , 33 : 17 – 33 .
   Web of Science ®Google Scholar
• Olmsted-Hawala , E. L. , Murphy , E. D. , Hawala , S. and Ashenfelter , K. T. 2010 . “ Think-aloud protocols: A comparison of three think-aloud protocols for use in testing data-dissemination web sites for usability ” . In Proceedings of the CHI 2010 Conference on Human Factors in Computing Systems , 2381 – 2390 . New York , NY : ACM Press .
   Google Scholar
• Pashler , H. 2000 . “ Task switching and multitask performance ” . In Control of cognitive processes: Attention and performance XVIII , Edited by: Monsell , S. and Driver , J. 277 – 307 . Cambridge , MA : MIT Press .
   Google Scholar
• Patrick , J. , Gregov , A. , Halliday , P. , Handley , J. and O‘Reilly , S. 1999 . Analysing operators' diagnostic reasoning during multiple events . Ergonomics , 42 : 493 – 515 .
   Web of Science ®Google Scholar
• Posner , M. I. and Nissen , M. J. 1976 . Visual dominance: An information-processing account of its origins and significance . Psychological Review , 83 : 157 – 171 .
   PubMed Web of Science ®Google Scholar
• Rubin , J. and Chisnell , D. 2008 . Handbook of usability testing: How to plan, design, and conduct effective tests , 2nd , Indianapolis , IN : Wiley .
   Google Scholar
• Russo , J. E. , Johnson , E. J. and Stephens , D. L. 1989 . The validity of verbal protocols . Memory & Cognition , 17 : 759 – 769 .
   PubMed Web of Science ®Google Scholar
• Salvucci , D. D. and Goldberg , J. H. 2000 . “ Identifying fixations and saccades in eye-tracking protocols ” . In Proceedings of the Eye Tracking Research & Applications Symposium , 71 – 78 . New York , NY : ACM Press .
   Google Scholar
• Sanderson , P. , Crawford , J. , Savill , A. , Watson , M. and Russell , W. J. 2004 . Visual and auditory attention in patient monitoring: A formative analysis . Cognition, Technology & Work , 6 : 172 – 185 .
   Google Scholar
• Schooler , J. W. 2002 . Verbalization produces a transfer inappropriate processing shift . Applied Cognitive Psychology , 16 : 989 – 997 .
   Web of Science ®Google Scholar
• Schooler , J. W. and Engstler-Schooler , T. Y. 1990 . Verbal overshadowing of visual memories: Some things are better left unsaid . Cognitive Psychology , 22 : 36 – 71 .
   PubMed Web of Science ®Google Scholar
• Seagull , F. J. , Wickens , C. D. and Loeb , R. G. 2001 . “ When is less more? Attention and workload in auditory, visual, and redundant patient-monitoring conditions ” . In Proceedings of the Human Factors and Ergonomics Society 45th Annual Meeting , 1395 – 1399 . Santa Monica , CA : HFES .
   Google Scholar
• Spencer , R. , Coiera , E. and Logan , P. 2004 . Variation in communication loads on clinical staff in the emergency department . Annals of Emergency Medicine , 44 : 268 – 273 .
   PubMed Web of Science ®Google Scholar
• Trafton , J. G. , Altmann , E. M. , Brock , D. P. and Mintz , F. E. 2003 . Preparing to resume an interrupted task: Effects of prospective goal encoding and retrospective rehearsal . International Journal of Human–Computer Studies , 58 : 583 – 603 .
   Web of Science ®Google Scholar
• Trafton , J. G. and Monk , C. A. 2007 . Task interruptions . Reviews of Human Factors and Ergonomics , 3 : 111 – 126 .
   Google Scholar
• Venturi , G. , Troost , J. and Jokela , T. 2006 . People, organizations, and processes: An inquiry into the adoption of user-centred design in industry . International Journal of Human–Computer Interaction , 21 : 219 – 238 .
   Web of Science ®Google Scholar
• Vredenburg , K. , Mao , J.-Y. , Smith , P. W. and Carey , T. 2002 . “ A survey of user-centered design practice ” . In Proceedings of the CHI 2002 Conference on Human Factors in Computing Systems , 471 – 478 . New York , NY : ACM Press .
   Google Scholar
• Vygotsky , L. S. 1988 . “ On inner speech ” . In Child language: A reader , Edited by: Franklin , M. B. and Barten , S. S. 181 – 187 . Oxford , , UK : Oxford University Press .
   Google Scholar
• Wilson , T. D. and Schooler , J. W. 1991 . Thinking too much: Introspection can reduce the quality of preferences and decisions . Journal of Personality and Social Psychology , 60 ( 2 ) : 181 – 192 .
   PubMed Web of Science ®Google Scholar
• Wright , R. B. and Converse , S. A. 1992 . “ Method bias and concurrent verbal protocol in software usability testing ” . In Proceedings of the Human Factors Society 36th Annual Meeting , 1220 – 1224 . Santa Monica , CA : HFS .
   Google Scholar