Using GOMS and NASA-TLX to Evaluate Human–Computer Interaction Process in Interactive Segmentation

References

• Abdulin, E. (2011). Using the keystroke-level model for designing user interface on middle-sized touch screens. Proceedings of the 2011 annual conference extended abstracts on Human factors in computing systems (pp. 673–686). Vancouver, Canada.
   Google Scholar
• Andrés, J. D. (2014). Towards an automatic user profiling system for online information sites Identifying demographic determining factors. Online Information Review, 39(1), 61–80.
   Web of Science ®Google Scholar
• Aquilab Artiview. (2016). A complete software platform for multimodality imaging, contouring, and evaluation in radiotherapy. Retrieved April 1, 2016, from http://www.aquilab.com/index.php/artiview.html
   Google Scholar
• Aselmaa, A., Goossens, R. H., Laprie, A., Ken, S., Fechter, T., Ramkumar, A., & Freudenthal, A. (2013). Workflow Analysis Report. Retrieved March 31, 2016, from http://summerproject.eu/work/deliverables/
   Google Scholar
• Balafar, M., Ramli, A., Saripan, M., & Mashohor, S. (2010). Review of brain MRI image segmentation methods. Artificial Intelligence Review, 33(3), 261–274.
   Web of Science ®Google Scholar
• Boykov, Y., & Jolly, M. P. (2001). Interactive graph cuts for optimal boundary and region segmentation of objects in N-D images. International Conference on Computer Vision (pp. 105–112). Vancouver, Canada.
   Google Scholar
• Bridger, R. S., & Brasher, K. (2011). Cognitive task demands, self-control demands and the mental well-being of office workers. Ergonomics, 54(9), 830–839.
   PubMed Web of Science ®Google Scholar
• Bruneau, D. P. J. (2006). Subjective mental workload assessment, International encyclopaedia of ergonomics and human factors (2nd ed., pp. 946–947). 1, USA: CRC press.
   Google Scholar
• Cain, B. (2007). A review of the mental workload literature (Report #RTOTR-HFM-121-part-II). Defense Research and Development, Toronto, Canada. Retrieved March 15, 2016, from http://www.dtic.mil/dtic/tr/fulltext/u2/a474193.pdf.
   Google Scholar
• Card, S., Moran, T., & Newell, A. (1983). The psychology of human-computer interaction. Hillsdale, NJ: Lawrence Erlbaum Associates. ISBN: 0898592437.
   Google Scholar
• Carmel, E., Crawford, S., & Chen, H. (1992). Browsing in hypertext: A cognitive study. IEEE Transactions on Systems, Man, and Cybernetics, 22(5), 865–884.
   Web of Science ®Google Scholar
• Chang, W., Hwang, W., & Ji, Y. G. (2011). Haptic seat interfaces for driver information and warning systems. International Journal of Human-Computer Interaction, 27(12), 1119–1132.
   Web of Science ®Google Scholar
• Christou, G., Ritter, F. E., & Jacob, R. J. K. (2012). Codein-A new notation for GOMS to handle evaluations of reality-based interaction style interfaces. International Journal of Human-Computer Interaction, 28(3), 189–201.
   Web of Science ®Google Scholar
• De Boer, R., Vrooman, H., Ikram, M., Vernooij, M., Breteler, M., Van de Lugt, A., & Niessen, W. (2010). Accuracy and reproducibility study of automatic MRI brain tissue segmentation methods. NeuroImage, 51(3), 1047–1056.
   PubMed Web of Science ®Google Scholar
• Dey, A., & Mann, D. D. (2010). Sensitivity and diagnosticity of NASA-TLX and simplified SWAT to assess the mental workload associated. Journal of Ergonomics, 53(7), 848–857.
   PubMed Web of Science ®Google Scholar
• Dirican, A. C., & Göktürk, M. (2011). Psychophysiological measures of human cognitive states applied in Human Computer Interaction. Procedia Computer Science, 3, 1361–1367.
   Google Scholar
• Dolz, J., Kirisli, H. A., Viard, R., & Massoptier, L. (2014a). Combining watershed and graph cuts methods to segment organs at risk in radiotherapy. Proc. SPIE 9034, Medical Imaging: Image Processing. San Diego, CA.
   Google Scholar
• Dolz, J., Kirisli, H. A., Viard, R., & Massoptier, L. (2014b). Interactive approach to segment organs at risk in radiotherapy treatment planning. Proc. SPIE 9034, Medical Imaging: Image Processing.
   Google Scholar
• Gao, Q., Wang, Y., Song, F., Li, Z., & Dong, X. (2015). Mental workload measurement for emergency operating procedures in digital nuclear power plants. Ergonomics, 56(7), 1070–1085.
   Web of Science ®Google Scholar
• Hammers, A., Chen, C. H., Lemieux, L., Allom, R., Vossos, S., Free, S. L., … Koepp, M. J. (2007). Statistical neuroanatomy of the human inferior frontal gyrus and probabilistic atlas in a standard stereotaxic space. Human Brain Mapping, 28, 34–48.
   PubMed Web of Science ®Google Scholar
• Harders, M., & Székely, G. (2003). Enhancing human – Computer interaction in medical segmentation. Proceedings of the IEEE, 91(9), 1430–1442.
   Web of Science ®Google Scholar
• Hart, S. G., & Staveland, L. E. (1988). Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In P. A. Hancock & N. Meshkati (Eds.), Human mental workload. Amsterdam, Netherlands: North-Holland Press.
   Google Scholar
• Heckel, F., Moltz, J. H., Tietjen, C., & Hahn, H. K. (2013). Sketch-based editing tools for tumour segmentation in 3D medical images. Computer Graphics Forum, 32(8), 144–157.
   Web of Science ®Google Scholar
• Jeon, M., & Croschere, J. (2015). Sorry, I’m late; I’m not in the mood: Negative emotions lengthen driving time. 12th International Conference on Engineering Psychology and Cognitive Ergonomics (Vol. 9174, pp. 237–244). Los Angeles, California.
   Google Scholar
• John, B. E., & Kieras, D. E. (1996). Using GOMS for user interface design and evaluation: Which technique? ACM Transactions on Computer-Human Interaction, 3(4), 287–319.
   Google Scholar
• Kang, Y., Engelke, K., & Kalender, W. A. (2004). Interactive 3D editing tools for image segmentation. Medical Image Analysis, 8, 35–46.
   PubMed Web of Science ®Google Scholar
• Karasev, P., Kolesov, I., Fritscher, K., Vela, P., Mitchell, P., & Tannenbaum, A. (2013). Interactive medical image segmentation using PDE control of active contours. IEEE Transactions on Medical Imaging, 32(11), 2127–2139.
   PubMed Web of Science ®Google Scholar
• Kieras, D. E. (1988). Towards a practical GOMS model methodology for user interface design. In M. Helander (Ed.), The handbook of human-computer interaction (pp. 135–158). North-Holland, Amsterdam: Elsevier.
   Google Scholar
• Kieras, D. E. (1993). Using the keystroke-level model to estimate execution times. Unpublished Report. University of Michigan. Retrieved March 31, 2016, from http://www.pitt.edu/~cmlewis/KSM.pdf
   Google Scholar
• Kieras, D. E. (1997a). A Guide to GOMS model usability evaluation using NGOMSL. In M. Helander, T. Landauer, & P. Prabhu (Eds.), The handbook of human-computer interaction (2nd ed., pp. 733–766). North-Holland, Amsterdam.
   Google Scholar
• Kotani, K., & Horii, K. (2003). An analysis of muscular load and performance in using a pen-tablet system. Journal of Physiological Anthropology, 22(2), 89–95.
   Google Scholar
• Li, J. Y., & Dang, J. W. (2012). Research and improvement of live-wire interactive algorithm for medical image segmentation. Applied Mechanics and Materials, 182–183, 1065–1068.
   Google Scholar
• Lin, C. J., Hsieh, T. L., & Lin, S. F. (2013). Development of staffing evaluation principle for advanced main control room and the effect on situation awareness and mental workload. Nuclear Engineering and Design, 265, 137–144.
   Web of Science ®Google Scholar
• Longo, L., & Kane, B. (2011, June 27–30). A novel methodology for evaluating user interfaces in health care. 24th International Symposium on Computer-Based Medical Systems(CBMS) (pp. 1–6). Briston, United Kingdom.
   Google Scholar
• Lundström, C., Rydell, T., Forsell, C., Persson, A., & Ynnerman, A. (2011). Multi-touch table system for medical visualization: Application to orthopedic surgery planning. IEEE Transactions on Visualization and Computer Graphics, 17(12), 1775–1784.
   PubMed Web of Science ®Google Scholar
• Mazur, L. M., Mosaly, P. R., Hoyle, L. M., Jones, E. L., Chera, B. S., & Marks, L. B. (2014). Relating physician’s workload with errors during radiation therapy planning. Practical Radiation Oncology, 4(2), 71–75.
   PubMedGoogle Scholar
• McGuinness, K., & O’Connor, N. E. (2010). A comparative evaluation of interactive segmentation algorithms. Pattern Recognition, 43(2), 434–444.
   Web of Science ®Google Scholar
• McGuinness, K., & O’Connor, N. E. (2011). Toward automated evaluation of interactive segmentation. Computer Vision and Image Understanding, 115(6), 868–884.
   Web of Science ®Google Scholar
• Miyake, S. (2001). Multivariate workload evaluation combining physiological and subjective measures. International Journal of Psychophysiology, 40(3), 233–238.
   PubMed Web of Science ®Google Scholar
• Morgan, J. F., & Hancock, P. A. (2011). The effect of prior task loading on mental workload: An example of hysteresis in driving. Human Factors: The Journal of the Human Factors and Ergonomics Society, 53(1), 75–86.
   PubMed Web of Science ®Google Scholar
• Mortensen, E. N., & Barrett, W. A. (1998). Interactive segmentation with intelligent scissors. Graphical Models and Image Processing, 60(5), 349–384.
   Google Scholar
• Mosaly, P. R., & Mazur, L. M. (2011). Empirical evaluation of workload of the radiation oncology physicist during radiation treatment planning and delivery. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 55(1), 753–775.
   Google Scholar
• Murata, A. (2006). Eye-gaze input versus mouse: Cursor control as a function of age. International Journal of Human-Computer Interaction, 21(1), 1–14.
   Web of Science ®Google Scholar
• Olabarriaga, S., & Smeulders, A. (2001). Interaction in the segmentation of medical images: A survey. Medical Image Analysis, 5(2), 127–142.
   PubMed Web of Science ®Google Scholar
• Oyewole, S. A., & Haight, J. M. (2011). Determination of optimal paths to task goals using expert system based on GOMS model. Computers in Human Behavior, 27, 823–833.
   Web of Science ®Google Scholar
• Petitjean, C., & Dacher, J. (2011). A review of segmentation methods in short axis cardiac MR images. Medical Image Analysis, 15, 169–184.
   PubMed Web of Science ®Google Scholar
• Ramkumar, A., Dolz, J., Kirisli, H. A., Varga, E., Schimek-Jasch, T., Nestle, U., … Song, Y. (2015). User interaction in semi-automatic segmentation of organs at Risk: A case study in radiotherapy. Journal of Digital Imaging, 29(2), 264–277.
   Web of Science ®Google Scholar
• Roscoe, A. H., & Ellis, G. A. (1990). A subjective rating scale for assessing pilot workload in flight: A decade of practical use (Technical report TR 90019). Retrieved April 1, 2016, from
   Google Scholar
• Rubio, S., Diaz, E., Martin, J., & Puente, J. (2004). Evaluation of subjective mental workload: A comparison of SWAT, NASA-TLX, and workload profile methods. Applied Psychology, 53(1), 61–86.
   Web of Science ®Google Scholar
• Saitwal, H., Feng, X., & Walji, M. (2010). Assessing performance of an electronic health record (EHR) using cognitive task analysis. International Journal of Medical Informatics, 79(7), 501–506.
   PubMed Web of Science ®Google Scholar
• Schrepp, M., & Fischer, P. A. (2006). GOMS model for keyboard navigation in web pages and web applications. Lecture Notes in Computer Science, 4061, 287–294.
   Google Scholar
• Sherbondy, A. J., Holmlund, D., Rubin, G. D., Schraedley, P. K., Winograd, T., & Napel, S. (2005). Alternative input devices for efficient navigation of large CT angiography data sets. Radiology, 234, 391–398.
   PubMed Web of Science ®Google Scholar
• Smelcer, J. B. (1995). User errors in database query composition. International Journal of Human–Computer Studies, 42, 353–381.
   Web of Science ®Google Scholar
• The Medical Imaging Interaction Toolkit (MITK). (2016). A toolkit facilitating the creation of interactive software by extending VTK and ITK. Retrieved April 1, 2016, from http://www.mitk.org
   Google Scholar
• Varian Medical Systems, Eclipse. (2016). Treatment Planning System. Retrieved April 1, 2016, from https://www.varian.com/oncology/products/software/treatment-planning/eclipse-proton
   Google Scholar
• Von Falck, C., Meier, S., Jördens, S., King, B., Galanski, M., & Shin, H. O. (2010). Semi-automated segmentation of pleural effusions in MDCT datasets. Academic Radiology, 17(7), 841–848.
   PubMed Web of Science ®Google Scholar
• West, R. L., & Nagy, G. (2007). Using GOMS for modeling routine tasks within complex sociotechnical systems: Connecting macrocognitive models to microcognition. Journal of Cognitive Engineering and Decision Making, 1(2), 186–211.
   Google Scholar
• Xiang, L. X. L., & Chen, X. L. (2010). The research on performance of automobile human-machine interface based on BHR-GOMS behavior model. Intelligent computer intelligent systems (ICIS), IEEE international Conference (Vol. 2, pp. 174–178). Xiamen, China.
   Google Scholar
• Yang, W., Cai, J., Zheng, J., & Luo, J. (2010). User-friendly interactive image segmentation through unified combinatorial user inputs. IEEE Transactions on Image Processing, 19(9), 2470–2479.
   PubMed Web of Science ®Google Scholar
• Zhao, F., & Xie, X. (2013). An overview of interactive medical image segmentation. Annals of BMVA, 2013(7), 1–22.
   Google Scholar