Ingredients and Meals Rather Than Recipes: A Proposal for Research That Does Not Treat Usability Evaluation Methods as Indivisible Wholes

REFERENCES

• Andre , T. S. , Hartson , H. R. , Belz , S. M. and McCreary , F. A. 2001 . The user action framework: a reliable foundation for usability engineering support tools . International Journal of Human-Computer Studies , 54 ( 1 ) : 107 – 136 .
   Web of Science ®Google Scholar
• Bailey , R. W. , Allan , R. W. and Raiello , P. Usability testing vs. heuristic evaluation: A head-to-head comparison . Proceedings of the Human Factors Society 36th Annual Meeting . pp. 409 – 413 .
   Google Scholar
• Barkhuus, L., & Rode, J. (2007, April). From mice to men—24 years of evaluation in CHI. Paper presented at the ACM CHI ′07—Alt.CHI, San Jose, CA. http://www.viktoria.se/altchi/ (http://www.viktoria.se/altchi/)
   Google Scholar
• Capra , M. and Smith-Jackson , T. 2005 . Developing guidelines for describing usability problems , Blacksburg : Virginia Tech, Assessment and Cognitive Ergonomics Laboratory & Human-Computer Interaction Laboratory . No. ACE/HCI-2005-002
   Google Scholar
• Chattratichart , J. and Lindgaard , G. 2008 . A comparative evaluation of heuristic-based usability inspection methods . CHI ′08 Extended Abstracts on Human Factors in Computing Systems , : 2213 – 2220 .
   Google Scholar
• Cockton , G. and Lavery , D. A framework for usability problem extraction . Proceedings of the IFIP TC.13 International Conference on Human-Computer Interaction . Vol. 9 , pp. 344 – 352 .
   Google Scholar
• Cockton , G. , Lavery , D. and Woolrych , A. 2008 . “ Inspection-based evaluations ” . In The human-computer interaction handbook , 2nd , Edited by: Sears , A. and Jacko , J. 1171 – 1190 . London, UK : CRC Press .
   Google Scholar
• Cockton , G. and Woolrych , A. 2001 . Understanding inspection methods: Lessons from an assessment of heuristic evaluation . People and Computers XV: Joint Proceedings of HCI 2001 and IHM , 2001 : 171 – 192 .
   Google Scholar
• Cockton, G., & Woolrych, A. (2009). Comparing UEMs: Strategies and implementation. Final report of COST-294 working group 2. http://141.115.28.2/cost294/upload/533.pdf (http://141.115.28.2/cost294/upload/533.pdf)
   Google Scholar
• Cockton , G. , Woolrych , A. , Hall , L. and Hindmarch , M. 2003 . Changing analysts' tunes: The surprising impact of a new instrument for usability inspection method assessment . People and Computers XVII: Designing for Society , : 145 – 162 .
   Google Scholar
• Cockton , G. , Woolrych , A. and Hindmarch , M. 2004 . Reconditioned merchandise: Extended structured report formats in usability inspection . CHI 2004 Extended Abstracts on Human Factors in Computer Systems , : 1433 – 1436 .
   Google Scholar
• Connell , I. W. and Hammond , N. V. Comparing usability evaluation principles with heuristics: Problem instances vs. problem types . Proceedings of the IFIP TC.13 International Conference on Human-Computer Interaction . pp. 621 – 629 .
   Google Scholar
• Courage , C. and Baxter , K. 2005 . Understanding your users: A practical guide to user requirements methods, tools, and techniques , San Francisco, CA : Morgan Kaufmann .
   Google Scholar
• Cuomo , D. L. and Bowen , C. D. Stages of user activity model as a basis for user-system interface evaluations . Proceedings of the Human Factors Society 36th Annual Meeting . pp. 1254 – 1258 .
   Google Scholar
• Dreyfus , H. and Dreyfus , S. 1986 . Mind over machine , New York, NY : Free Press .
   Google Scholar
• Feyerabend , P. 1975 . Against method: Outline of an anarchistic theory of knowledge , London, UK : Humanities .
   Google Scholar
• Frøkjær , E. and Hornbæk , K. 2008 . Metaphors of human thinking for usability inspection and design . ACM Transactions on Computer-Human Interaction , 14 ( 4 ) : 20
   Google Scholar
• Furniss , D. 2008 . Beyond problem identification: Valuing methods in a ‘system of usability practice’ , London, UK : University College London .
   Google Scholar
• Furniss , D. , Blandford , A. and Curzon , P. 2008 . “ Usability work in professional website design: Insights from practitioners' perspectives ” . In Maturing usability: Quality in software, interaction and value , Edited by: Law , E. , Hvannberg , E. and Cockton , G. 144 – 167 . New York, NY : Springer .
   Google Scholar
• Gilbert , J. D. , Williams , A. and Seals , C. D. 2007 . Clustering for usability participant selection . Journal of Usability Studies , 3 ( 1 ) : 40 – 52 .
   Google Scholar
• Gray , W. D. and Salzman , M. C. 1998 . Damaged merchandise? A review of experiments that compare usability evaluation methods . Human-Computer Interaction , 13 : 203 – 261 .
   Web of Science ®Google Scholar
• Gulliksen , J. , Boivie , I. and Göransson , B. 2006 . Usability professionals—Current practices and future development . Interacting with Computers , 18 : 568 – 600 .
   Web of Science ®Google Scholar
• Hertzum , M. , Hansen , K. D. and Andersen , H. H. K. 2009 . Scrutinising usability evaluation: does thinking aloud affect behaviour and mental workload? . Behaviour & Information Technology , 28 : 165 – 181 .
   Web of Science ®Google Scholar
• Hertzum , M. and Jacobsen , N. E. 2001 . The evaluator effect: A chilling fact about usability evaluation methods . International Journal of Human-Computer Interaction , 13 : 421 – 443 .
   Web of Science ®Google Scholar
• Holtzblatt , K. , Wendell , J. B. and Wood , S. 2005 . Rapid contextual design: A how-to guide to key techniques for user-centered design , San Francisco, CA : Morgan Kaufmann .
   Google Scholar
• Hornbæk , K. 2010 . Dogmas in the assessment of usability evaluation methods . Behaviour & Information Technology , 29 ( 1 ) : 97 – 111 .
   Web of Science ®Google Scholar
• Hornbæk , K. and Frøkjær , E. 2004 . Usability inspection by metaphors of human thinking compared to heuristic evaluation . International Journal of Human–Computer Interaction , 17 : 357 – 374 .
   Web of Science ®Google Scholar
• Hornbæk , K. and Frøkjær , E. What kinds of usability-problem description are useful to developers? . Proceedings of the Human Factors and Ergonomic Society's Annual Meeting . pp. 2523 – 2527 .
   Google Scholar
• Hornbæk , K. and Frøkjær , E. 2008a . Comparison of techniques for matching of usability problem descriptions . Interacting with Computers , 20 : 505 – 514 .
   Web of Science ®Google Scholar
• Hornbæk , K. and Frøkjær , E. Making use of business goals in usability evaluation: an experiment with novice evaluators . Proceedings of the ACM Conference on Human Factors in Computing Systems . pp. 903 – 912 .
   Google Scholar
• Howarth , J. , Andre , T. S. and Hartson , R. 2007 . A structured process for transforming usability data into usability information . Journal of Usability Studies , 3 ( 1 ) : 7 – 23 .
   Google Scholar
• Hvannberg , E. T. , Law , E. L.-C. and Lárusdóttir , M. K. 2007 . Heuristic evaluation: Comparing ways of finding and reporting usability problems . Interacting with Computers , 19 : 225 – 240 .
   Web of Science ®Google Scholar
• Jacobsen , N. E. and John , B. E. Two case studies in using cognitive walkthroughs for interface evaluation . Tech. Rep. No. CMU-CS-00-132 . 2000 . Pittsburgh, PA: Carnegie Mellon University School of Computer Science
   Google Scholar
• Jeffries , R. 1994 . “ Usability problem reports: Helping evaluators communicate effectively with developers ” . In Usability inspection methods , Edited by: Nielsen , J. and Mack , R. L. 273 – 294 . New York, NY : Wiley .
   Google Scholar
• Jeffries , R. , Miller , J. , Wharton , C. and Uyeda , K. User interface evaluation in the real world: A comparison of four techniques . Proceedings of ACM Conference on Human Factors in Computer Systems . pp. 119 – 124 .
   Google Scholar
• John , B. E. and Packer , H. Learning and using the cognitive walkthrough method: a case study approach . Proceedings of ACM Conference on Human Factors in Computer Systems . pp. 429 – 436 .
   Google Scholar
• Jones , J. C. 1988 . “ Softecnica ” . In Design after modernism: Beyond the object , Edited by: Thackera , J. 216 – 266 . London, UK : Thames & Hudson .
   Google Scholar
• Jones , J. C. 1992 . Design methods: Seeds of human futures , New York, NY : Wiley .
   Google Scholar
• Kieras , D. 1988 . “ Towards a practical GOMS model methodology for user interface design ” . In Handbook of human-computer interaction , Edited by: Helander , M. 135 – 157 . Amsterdam, , the Netherlands : Elsevier Science .
   Google Scholar
• Kjeldskov , J. , Skov , M. and Stage , J. Instant data analysis . Proceedings of the Third Nordic Conference on Human-Computer Interaction . pp. 233 – 240 .
   Google Scholar
• Lavery , D. and Cockton , G. Representing predicted and actual usability problems . Proceedings of the International Workshop on Representations in Interactive Software Development . pp. 97 – 108 .
   Google Scholar
• Lavery, D., Cockton, G., & Atkinson, M. P. (1996). Heuristic evaluation: Usability evaluation materials (Tech. Rep. TR-1996-15). Glasgow, Scotland: University of Glasgow. http://www.dcs.gla.ac.uk/asp/materials/HE_1.0/materials.pdf (http://www.dcs.gla.ac.uk/asp/materials/HE_1.0/materials.pdf)
   Google Scholar
• Lavery , D. , Cockton , G. and Atkinson , M. P. 1997 . Comparison of evaluation methods using structured usability problem reports . Behaviour and Information Technology , 16 : 246 – 266 .
   Web of Science ®Google Scholar
• Law , L. and Hvannberg , E. T. Complementarity and convergence of heuristic evaluation and usability test: a case study of universal brokerage platform . Proceedings of the Second Nordic Conference on Human-Computer interaction . New York, NY. pp. 71 – 80 . ACM .
   Google Scholar
• Law , E. and Hvannberg , E. T. Consolidating usability problems with novice evaluators . Proceedings of the 5th Nordic Conference on Human-Computer interaction . New York, NY. pp. 495 – 498 . ACM .
   Google Scholar
• Lindgaard , G. and Chattratichart , J. Usability testing: what have we overlooked? . Proceedings of the ACM Conference on Human Factors in Computing Systems . pp. 1415 – 1424 .
   Google Scholar
• Mankoff , J. , Dey , A. K. , Hsieh , G. , Kientz , J. , Ames , M. and Lederer , S. Heuristic evaluation of ambient displays . Proceedings of the ACM Conference on Human Factors in Computing Systems . pp. 169 – 176 .
   Google Scholar
• Mathiasen , L. 1982 . Systems develpment and systems development method , Aarhus, Denmark : Aarhus University . Unpublished doctoral dissertation
   Google Scholar
• Mayhew , D. J. 1999 . The usability engineering lifecycle , San Francisco, CA : Morgan Kaufmann .
   Google Scholar
• Molich , R. , Ede , M. R. , Kaasgaard , K. and Karyukin , B. 2004 . Comparative usability evaluation . Behaviour & Information Technology , 23 ( 1 ) : 65 – 74 .
   Web of Science ®Google Scholar
• Molich , R. and Nielsen , J. 1990 . Improving a human-computer dialogue . Communications of the ACM , 33 : 338 – 348 .
   Web of Science ®Google Scholar
• Naur , P. 1995 . Knowing and the mystique of logic and rules , Dordrecht, , the Netherlands : Kluwer Academic .
   Google Scholar
• Nielsen , J. 1994 . “ Heuristic evaluation ” . In Usability inspection methods , Edited by: Nielsen , J. and Mack , R. 25 – 62 . New York, NY : Wiley .
   Google Scholar
• Nielsen, J. (n.d.). How to conduct a heuristic evaluation. http://www.useit.com/papers/heuristic/heuristic_evaluation.html (http://www.useit.com/papers/heuristic/heuristic_evaluation.html)
   Google Scholar
• Nielsen , J. and Molich , R. 1990 . Heuristic evaluation of user interfaces . ACM Conference on Human Factors in Computing , : 249 – 256 .
   Google Scholar
• Nørgaard , M. and Hornbæk , K. What do usability evaluators do in practice? An explorative study of think-aloud testing . ACM Conference on Designing Interactive Systems . pp. 209 – 218 .
   Google Scholar
• Okasha , S. 2002 . Philosophy of science: A very short introduction , Oxford, UK : Oxford Paperbacks .
   Google Scholar
• Rieman , J. , Davies , S. , Hair , D. C. , Esemplare , M. , Polson , P. and Lewis , C. An automated cognitive walkthrough . Proceedings of the ACM Conference on Human Factors in Computing Systems . pp. 427 – 428 .
   Google Scholar
• Rosenbaum , S. 2008 . “ The future of usability evaluations: Increasing impact on value ” . In Maturing usability: Quality in software, interaction and value , Edited by: Law , E. L. C. , Hvannberg , E. and Cockton , G. 344 – 378 . London, UK : Springer .
   Google Scholar
• Sears , A. 1997 . Heuristic walkthroughs: Finding the problems without the noise . International Journal of Human-Computer Interaction , 9 : 213 – 234 .
   Web of Science ®Google Scholar
• Sears , A. and Hess , D. 1999 . Cognitive walkthroughs: Understanding the effect of task description detail on evaluator performance . International Journal of Human-Computer Interaction , 11 : 185 – 200 .
   Web of Science ®Google Scholar
• Skov , M. and Stage , J. Supporting problem identification in usability evalautions . Proceedings of the Australian Computer-Human Interaction Conference (OZCHI) . pp. 1 – 9 .
   Google Scholar
• Somervell , J. and McCrickard , D. S. 2005 . Better discount evaluation: illustrating how critical parameters support heuristic creation . Interacting with Computers , 17 : 592 – 612 .
   Web of Science ®Google Scholar
• Sorell , T. 2001 . Descartes , Oxford, UK : Oxford University Press .
   Google Scholar
• Spencer , R. The streamlined cognitive walkthrough method, working around social constraints encountered in a software development company . Proceedings of the ACM Conference on Human Facors in Comuting Systems . pp. 353 – 359 .
   Google Scholar
• Spool , J. and Shroeder , W. 2001 . Testing web sites: Five users is nowhere near enough . CHI ′01 Extended Abstracts on Human Factors in Computing Systems , : 285 – 286 .
   Google Scholar
• Suchman , L. 1987 . Plans and situated actions: The problem of human-machine communication , New York, NY : Cambridge University Press .
   Google Scholar
• Sy , D. 2007 . Adapting usability investigations for agile user-centered design . Journal of Usability Studies , 2 : 112 – 132 .
   Google Scholar
• Theofanos , M. and Quesenbery , W. 2005 . Towards the design of effective formative test reports . Journal of Usability Studies , 1 ( 1 ) : 27 – 45 .
   Google Scholar
• Uldall-Espersen , T. , Frøkjær , E. and Hornbæk , K. 2008 . Tracing impact in a usability improvement process . Interacting with Computers , 20 ( 1 ) : 48 – 63 .
   Google Scholar
• Vermeeren, A. P. O. S. (2009). What's the problem? Studies on identifying usability problems in user tests. Unpublished doctoral dissertation, Delft University of Technology, Faculty of Industrial Design Engineering, Delft, the Netherlands. http://homepages.ipact.nl/~vermeeren/PhDThesis%20and%20Propositions%20APOSVermeeren-Whats%20the%20problem.pdf (http://homepages.ipact.nl/~vermeeren/PhDThesis%20and%20Propositions%20APOSVermeeren-Whats%20the%20problem.pdf)
   Google Scholar
• Wharton , C. , Rieman , J. , Lewis , C. and Polson , P. 1994 . “ The cognitive walkthrough method: A practitioner's guide ” . In Usability inspection methods , Edited by: Nielsen , J. and Mack , R. L. 105 – 140 . New York, NY : Wiley .
   Google Scholar
• Wixon , D. 2003 . Evaluating usability methods: Why the current literature fails the practitioner . interactions , 10 ( 4 ) : 29 – 34 .
   Google Scholar
• Woolrych , A. and Cockton , G. Why and when five test users aren't enough . Proceedings of IHM-HCI 2001 Conference . Toulouse, France. Edited by: Vanderdonckt , J. , Blandford , A. and Derycke , A. Vol. 2 , pp. 105 – 108 . Cépadèus Éditions .
   Google Scholar
• Woolrych , A. , Cockton , G. and Hindmarch , M. 2005 . Knowledge resources in usability inspection . British HCI Conference , 2 : 5 – 20 .
   Google Scholar