Psychometric Evaluation of the F-SUS: Creation and Validation of the French Version of the System Usability Scale

References

• Adinda, P. P., & Suzianti, A. (2018). Redesign of user interface for E-government application using usability testing method. In ICCIP (pp. 145–149). Qingdao, China. https://doi.org/10.1145/3290420.3290433
   Google Scholar
• Aguilar-Reyes, F., Poblete, K. L., & González, V. M. (2015). UX-AdChat: Semi-automatic, predictable usability evaluator of written dialogue systems. In CLIHC. Córdoba, Argentina.
   Google Scholar
• AlGhannam, B. A., Albustan, S. A., Al-Hassan, A. A., & Albustan, L. A. (2018). Towards a standard arabic system usability scale: psychometric evaluation using communication disorder app. International Journal of Human-Computer Interaction, 34(9), 799–804. https://doi.org/10.1080/10447318.2017.1388099
   Web of Science ®Google Scholar
• Ašeriškis, D., & Damaševičius, R. (2014). Gamification of a project management system. In ACHI 2014-7th International Conference on Advances in Computer-Human Interactions (pp. 200–207). Barcelona, Spain.
   Google Scholar
• Bangor, A., Kortum, P., & Miller, J. (2009). Determining what individual SUS scores mean: Adding an adjective rating scale. Journal of Usability Studies, 4(3), 114–123. https://uxpajournal.org/determining-what-individual-sus-scores-mean-adding-an-adjective-rating-scale/
   Google Scholar
• Bangor, A., Kortum, P. T., & Miller, J. T. (2008). An empirical evaluation of the system usability scale. International Journal of Human-Computer Interaction, 24(6), 574–594. https://doi.org/10.1080/10447310802205776
   Web of Science ®Google Scholar
• Baumgartner, J., Frei, N., Kleinke, M., Sauer, J., & Sonderegger, A. (2019). Pictorial system usability scale (P-SUS): Developing an instrument for measuring perceived usability. In CHI 2019, May 4–9,2019, Glasgow, Scotland, UK (pp. 1–11). https://doi.org/10.1145/3290605.3300299
   Google Scholar
• Berke, L., Huenerfauth, M., & Patel, K. (2017). Design and psychometric evaluation of American sign language translations of usability questionnaires. In ASSETS’17, Oct. 29–Nov. 1, 2017, Baltimore, MD, USA (pp. 175–184). https://doi.org/10.1145/3314205
   Google Scholar
• Berkman, M. I., & Karahoca, D. (2016). Re-Assessing the usability metric for user experience (UMUX) scale. Journal of Usability Studies, 11(3), 89–109. https://uxpajournal.org/assessing-usability-metric-umux-scale/
   Web of Science ®Google Scholar
• Beul-Leusmann, S., Samsel, C., Wiederhold, M., Krempels, K. H., Jakobs, E. M., & Ziefle, M. (2014). Usability evaluation of mobile passenger information systems. In Marcus A. (Eds.), Design, user experience, and usability. Theories, methods, and tools for designing the user experience. DUXU 2014. Lecture notes in computer science (Vol. 8517). Springer, Cham. https://doi.org/10.1007/978-3-319-07668-3_22
   Google Scholar
• Blažica, B., & Lewis, J. R. (2015). A Slovene translation of the system usability scale: The SUS-SI. International Journal of Human-Computer Interaction, 31(2), 112–117. https://doi.org/10.1080/10447318.2014.986634
   Web of Science ®Google Scholar
• Blecken, A., Brüggemann, D., & Marx, W. (2010). Usability evaluation of a learning management system. In Proceedings of the Annual Hawaii International Conference on System Sciences (pp. 1–9). Honolulu, HI, USA. https://doi.org/10.1109/HICSS.2010.422
   Google Scholar
• Bolchini, D., & Garzotto, F. (2008). Quality and potential for adoption of usability evaluation methods: An empirical study on MiLE+. Journal of Web Engineering, 7(4), 299–317. http://dl.acm.org/citation.cfm?id=2011277.2011281
   Web of Science ®Google Scholar
• Borkowska, A., & Jach, K. (2017). Pre-testing of polish translation of system usability scale (SUS). In Borzemski L., Grzech A., Świątek J., Wilimowska Z. (Eds.), Information systems architecture and technology: Proceedings of 37th international conference on information systems architecture and technology – ISAT 2016 – Part I. Advances in intelligent systems and computing (Vol. 521). Springer, Cham. https://doi.org/10.1007/978-3-319-46583-8_12
   Google Scholar
• Borsci, S., Federici, S., & Lauriola, M. (2009). On the dimensionality of the system usability scale: A test of alternative measurement models. Cognitive Processing, 10(3), 193–197. https://doi.org/10.1007/s10339-009-0268-9
   PubMed Web of Science ®Google Scholar
• Botella, F., Moreno, J. P., & Peñalver, A. (2014). How efficient can be a user with a tablet versus a smartphone? In Interaccion’14 (Vol. 10-12-Sept). Puerto de la Cruz, Tenerife, Spain. https://doi.org/10.1145/2662253.2662317
   Google Scholar
• Braunhofer, M., Elahi, M., & Ricci, F. (2014). Usability assessment of a context-aware and personality-based mobile recommender system. In Hepp M., Hoffner Y. (Eds.), E-Commerce and web technologies. EC-web 2014. Lecture notes in business information processing (Vol. 188). Springer, Cham. https://doi.org/10.1007/978-3-319-10491-1_9
   Google Scholar
• Brooke. (1996). SUS: A ‘quick and dirty’ usability scale. In P. W. Jordan, B. Thomas, B. A. Weerdmeester, & I. McClelland Eds., Usability evaluation in industry, 189–194 Taylor & Francis. http://hell.meiert.org/core/pdf/sus.pdf
   Google Scholar
• Brooke, J. (2013). SUS: A retrospective. Journal of Usability Studies, 8(2), 29–40. http://www.usabilityprofessionals.org/upa_publications/jus/2013february/brooke1.html%5Cnhttp://www.usability.gov/how-to-and-tools/methods/system-usability-scale.html
   Google Scholar
• Cattell, R. B. (1966). The scree test for the number of factors. Multivariate Behavioral Research, 1(2), 245–276. https://doi.org/10.1207/s15327906mbr0102_10
   PubMed Web of Science ®Google Scholar
• Chin, J. P., Diehl, V. A., & Norman, L. K. (1988). Development of an instrument measuring user satisfaction of the human-computer interface. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems - CHI ’88, 213–218. Washington, D.C., USA. https://doi.org/10.1145/57167.57203
   Google Scholar
• Çöltekin, A., Heil, B., Garlandini, S., & Fabrikant, S. I. (2009). Evaluating the effectiveness of interactive map interface designs: A case study integrating usability metrics with eye-movement analysis. Cartography and Geographic Information Science, 36(1), 5–17. https://doi.org/10.1559/152304009787340197
   Web of Science ®Google Scholar
• Dhillon, J. S., Wünsche, B. C., & Lutteroth, C. (2012). Evaluation of a web-based telehealth system: A preliminary investigation with seniors in New Zealand. In CHINZ’12 (pp.17–24). Dunedin, New Zealand. https://doi.org/10.1145/2379256.2379260
   Google Scholar
• Dianat, I., Ghanbari, Z., & AsghariJafarabadi, M. (2014). Psychometric properties of the persian language version of the system usability scale. Health Promotion Perspectives, 4(1), 82–829. https://doi.org/10.5681/hpp.2014.011
   PubMedGoogle Scholar
• Drost, E. A. (2011). Validity and reliability in social science research. Education Research and Perspectives, 38(1), 105–123. https://www.erpjournal.net/wp-content/uploads/2020/02/ERPV38-1.-Drost-E.-2011.-Validity-and-Reliability-in-Social-Science-Research.pdf
   Google Scholar
• Evans, J., Papadopoulos, A., Silvers, C. T., Charness, N., Boot, W. R., Schlachta-Fairchild, L., … Ent, C. B. (2016). Remote health monitoring for older adults and those with heart failure: Adherence and system usability. Telemedicine and E-Health, 22(6), 480–488. https://doi.org/10.1089/tmj.2015.0140
   PubMed Web of Science ®Google Scholar
• Finstad, K. (2010). The usability metric for user experience. Interacting with Computers, 22(5), 323–327. https://doi.org/10.1016/j.intcom.2010.04.004
   Web of Science ®Google Scholar
• Fritz, F., Balhorn, S., Riek, M., Breil, B., & Dugas, M. (2012). Qualitative and quantitative evaluation of EHR-integrated mobile patient questionnaires regarding usability and cost-efficiency. International Journal of Medical Informatics, 81(5), 303–313. https://doi.org/10.1016/j.ijmedinf.2011.12.008
   PubMed Web of Science ®Google Scholar
• García-Peñalvo, F. J., García-Holgado, A., Vázquez-Ingelmo, A., & Seoane-Pardo, A. M. (2018). Usability test of WYRED platform. In P. Zaphiris & A. Ioannou, (Eds.), Learning and collaboration technologies. Design, development and technological innovation. LCT 2018. Lecture notes in computer science (Vol. 10924). Springer, Cham. https://doi.org/10.1007/978-3-319-91743-6_5
   Google Scholar
• Georgsson, M., & Staggers, N. (2016). Quantifying usability: An evaluation of a diabetes mHealth system on effectiveness, efficiency, and satisfaction metrics with associated user characteristics. Journal of the American Medical Informatics Association, 23(1), 5–11. https://doi.org/10.1093/jamia/ocv099
   PubMed Web of Science ®Google Scholar
• Ghosh, D., Foong, P. S., Zhang, S., & Zhao, S. (2018). Assessing the utility of the system usability scale for evaluating voice-based user interfaces. In ChineseCHI’18 (pp. 11–15). Montreal, QC, Canada. https://doi.org/10.1145/3202667.3204844
   Google Scholar
• Hafiz, P., Maharjan, R., & Kumar, D. (2018). Usability of a mood assessment smartphone prototype based on humor appreciation. In MobileHCI 2018 - Beyond Mobile: The Next 20 Years - 20th International Conference on Human-Computer Interaction with Mobile Devices and Services, Conference Proceedings Adjunct (pp. 151–157). Barcelona, Spain. https://doi.org/10.1145/3236112.3236134
   Google Scholar
• Harrati, N., Bouchrika, I., Tari, A., & Ladjailia, A. (2016). Exploring user satisfaction for e-learning systems via usage-based metrics and system usability scale analysis. Computers in Human Behavior, 61(August), 463–471. https://doi.org/10.1016/j.chb.2016.03.051
   Web of Science ®Google Scholar
• Hassenzahl, M., Burmester, M., & Koller, F. (2003). AttrakDiff: Ein Fragebogen zur Messung wahrgenommener hedonischer und pragmatischer Qualität. In G. Szwillus & J. Ziegler, (Eds.), Mensch & Computer 2003: Interaktion in Bewegung (pp. 187–196). Stuttgart: B. G. Teubner. https://doi.org/10.1007/978-3-322-80058-9_19
   Google Scholar
• Heinonen, R., Luoto, R., Lindfors, P., & Nygård, C. H. (2012). Usability and feasibility of mobile phone diaries in an experimental physical exercise study. Telemedicine and E-Health, 18(2), 115–119. https://doi.org/10.1089/tmj.2011.0087
   PubMed Web of Science ®Google Scholar
• Held, J. P., Klaassen, B., Van Beijnum, B.-J. F., Luft, A. R., & Veltink, P. H. (2017). Usability evaluation of a vibrotactile feedback system in stroke subjects. Frontiers in Bioengineering and Biotechnology, 4(January), 1–7. https://doi.org/10.3389/fbioe.2016.00098
   Google Scholar
• Hookham, G., Nesbitt, K., & Kay-Lambkin, F. (2016). Comparing usability and engagement between a serious game and a traditional online program. ACM International Conference Proceeding Series, 01-05-Febr. Canberra, Australia. https://doi.org/10.1145/2843043.2843365
   Google Scholar
• Hornbaek, K. (2006). Current practice in measuring usability: Challenges to usability studies and research. International Journal of Human-Computer Studies, 64(2), 79–102. https://doi.org/10.1016/j.ijhcs.2005.06.002
   Web of Science ®Google Scholar
• Hussain, A., Mkpojiogu, E. O. C., & Hussain, Z. (2015). Usability evaluation of a web-based health awareness portal on smartphone devices using ISO 9241-11 model. Jurnal Teknologi, 77(4), 1–5. https://doi.org/10.11113/jt.v77.6035
   Web of Science ®Google Scholar
• ISO 13407:1999.(1999). Human-centred design processes for interactive systems (ISO 13407:1999). International organization for standardization. http://scholar.google.com/scholar?hl=en&btnG=Search&q=intitle:Human-centred+design+processes+for+interactive+systems+(ISO+13407:1999)#5
   Google Scholar
• ISO 9241-11:2018. (2018). Ergonomics of human-system interaction — Part 11: Usability: Definitions and concepts.
   Google Scholar
• ISO 9241-210:2010. (2010). Ergonomics of human-system interaction - Part 210: Human-centred design for interactive systems.
   Google Scholar
• Karlin, B., & Ford, R. (2013). The usability perception scale (UPscale): A measure for evaluating feedback displays. In Marcus A. (Eds.), Design, user experience, and usability. Design philosophy, methods, and tools. DUXU 2013. Lecture notes in computer science (Vol. 8012). Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39229-0
   Google Scholar
• Katsanos, C., Tselios, N., & Xenos, M. (2012). Perceived usability evaluation of learning management systems: A first step towards standardization of the system usability scale in Greek. In Proceedings of the 2012 16th Panhellenic Conference on Informatics, PCI 2012, Piraeus, Greece: IEEE CPS, 5-7 Oct., 2012 (pp. 302–307). Piraeus, Greece. https://doi.org/10.1109/PCi.2012.38
   Google Scholar
• Kerzazi, N., & Lavallee, M. (2011). Inquiry on usability of two software process modeling systems using ISO/IEC 9241. In Canadian Conference on Electrical and Computer Engineering (pp. 000773–000776). Niagara Falls, ON, Canada. https://doi.org/10.1109/CCECE.2011.6030560
   Google Scholar
• Kim, M. S., Shapiro, J. S., Genes, N., Aguilar, M. V., Mohrer, D., Baumlin, K., & Belden, J. L. (2012). A pilot study on usability analysis of emergency department information system by nurses. Applied Clinical Informatics, 3(1), 135–153. https://doi.org/10.4338/ACI-2011-11-RA-0065
   PubMed Web of Science ®Google Scholar
• Kirakowski, J., & Cierlik, B. (1998). Measuring the usability of web sites. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 42(4), 424–428. https://doi.org/10.1177/154193129804200405
   Google Scholar
• Kirakowski, J., & Corbett, M. (1993). SUMI: The software usability measurement inventory. British Journal of Educational Technology, 24(3), 210–212. http://onlinelibrary.wiley.com/doi/10.1111/j.1467-8535.1993.tb00076.x/abstract
   Web of Science ®Google Scholar
• Kline, R. B. (2005). Principles and practice of structural equation modeling ((2nd ed.). The Guilfo.
   Google Scholar
• Kortum, P., Acemyan, C. Z., & Oswald, F. L. (2020). Is it time to go positive? Assessing the positively worded system usability scale (SUS). Human Factors. https://doi.org/10.1177/0018720819881556
   Web of Science ®Google Scholar
• Kortum, P., & Oswald, F. L. (2018). The impact of personality on the subjective assessment of usability. International Journal of Human-Computer Interaction, 34(2), 177–186. https://doi.org/10.1080/10447318.2017.1336317
   Web of Science ®Google Scholar
• Kortum, P., & Sorber, M. (2015). Measuring the Usability of mobile applications for phones and tablets. International Journal of Human-Computer Interaction, 31(8), 518–529. https://doi.org/10.1080/10447318.2015.1064658
   Web of Science ®Google Scholar
• Kortum, P. T., & Bangor, A. (2013). Usability ratings for everyday products measured with the system usability scale. International Journal of Human-Computer Interaction, 29(2), 67–76. https://doi.org/10.1080/10447318.2012.681221
   Web of Science ®Google Scholar
• Lacerda, T. C., Von Wangenheim, C. G., Von Wangenheim, A., & Giuliano, I. (2014). Does the use of structured reporting improve usability? A comparative evaluation of the usability of two approaches for findings reporting in a large-scale telecardiology context. Journal of Biomedical Informatics, 52, 222–230. https://doi.org/10.1016/j.jbi.2014.07.002
   PubMed Web of Science ®Google Scholar
• Lallemand, C., Koenig, V., Gronier, G., & Martin, R. (2015). Création et validation d’une version française du questionnaire AttrakDiff pour l’évaluation de l’expérience utilisateur des systèmes interactifs. Revue Européenne De Psychologie Appliquée/European Review of Applied Psychology, 65(5), 239–252. https://doi.org/10.1016/j.erap.2015.08.002
   Google Scholar
• Landauer, T. K. (1997). Chapter 9 - behavioral research methods in human-computer interaction. In Handbook of human-computer interaction (Second ed., pp. 203–227). North-Holland.
   Google Scholar
• Larue, V. (2009). Apport du System Usability Scale à l’activité d’ergonomie d’évaluation. In IHM 2009, 13-16 Octobre 2009, Grenoble, France (pp. 155–161).
   Google Scholar
• Laugwitz, B., Held, T., & Schrepp, M. (2008). Construction and evaluation of a user experience questionnaire. USAB 2008 (pp. 63–76). https://doi.org/10.1007/978-3-540-89350-9_6
   Google Scholar
• Lewis, J. (1995). IBM Computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use. International Journal of Human‐Computer Interaction, 7(1), 57–78. http://www.tandfonline.com/doi/abs/10.1080/10447319509526110
   Web of Science ®Google Scholar
• Lewis, J., Utesch, B., & Maher, D. (2013). UMUX-LITE: When there’s no time for the SUS. In Proceedings of the SIGCHI Conference on Human Computer Interaction (pp. 2099–2102). Paris, France. http://dl.acm.org/citation.cfm?id=2481287
   Google Scholar
• Lewis, J. R. (2018). The system usability scale: Past, present, and future. International Journal of Human-Computer Interaction, 34(7), 577–590. https://doi.org/10.1080/10447318.2018.1455307
   Web of Science ®Google Scholar
• Lewis, J. R., & Sauro, J. (2009). The factor structure of the system usability scale. In Human Centered Design, First International Conference, HCD 2009, Held as Part of HCI International 2009, San Diego, CA, USA, July 19-24 (pp. 26–31). https://doi.org/10.1007/978-3-642-02806-9
   Google Scholar
• Lewis, J. R., & Sauro, J. (2017). Revisiting the factor structure of the system usability scale. Journal of Usability Studies, 12(4), 183–192. https://uxpajournal.org/revisit-factor-structure-system-usability-scale/
   Web of Science ®Google Scholar
• Liang, J., Xian, D., Liu, X., Fu, J., Zhang, X., Tang, B., & Lei, J. (2018). Usability study of mainstream wearable fitness devices: Feature analysis and system usability scale evaluation. JMIR MHealth and UHealth, 6(11), e11066. https://doi.org/10.2196/11066
   Web of Science ®Google Scholar
• Lin, H., Choong, -Y.-Y., & Salvendy, G. (1997). A proposed index of usability: A method for comparing the relative usability of different software systems. Behaviour & Information Technology, 16(4), 267–277. https://doi.org/10.1080/014492997119833
   Web of Science ®Google Scholar
• Lin, H. C. K., Chen, M. C., & Chang, C. K. (2015). Assessing the effectiveness of learning solid geometry by using an augmented reality-assisted learning system. Interactive Learning Environments, 23(6), 799–810. https://doi.org/10.1080/10494820.2013.817435
   Web of Science ®Google Scholar
• Lin, H. C. K., Wu, C. H., & Hsueh, Y. P. (2014). The influence of using affective tutoring system in accounting remedial instruction on learning performance and usability. Computers in Human Behavior, 41(December), 514–522. https://doi.org/10.1016/j.chb.2014.09.052
   Web of Science ®Google Scholar
• Lin, H. K., Hsieh, M., Wang, C., Sie, Z., & Chang, S. (2011). Establishment and usability evaluation of an interactive ar learning system on conservation of fish. TOJET: The Turkish Online Journal of Educational Technology, 10(4), 181–187. http://www.tojet.net/articles/v10i4/10418.pdf
   Google Scholar
• Lund, A. M. (2001). Measuring usability with the USE questionnaire. Usability Interface, 8(2), 3–6. https://doi.org/10.1177/1078087402250360
   Google Scholar
• Lutes, K. D., Chang, K., & Baggili, I. M. (2006). Diabetic e-management system (DEMS). In Proceedings - Third International Conference onInformation Technology: New Generations, ITNG 2006 (Vol.2006, pp. 619–624). Las Vegas, NV, USA. https://doi.org/10.1109/ITNG.2006.53
   Google Scholar
• Maguire, M. (2001). Methods to support human-centred design. International Journal of Human-Computer Studies, 55(4), 587–634. https://doi.org/10.1006/ijhc.2001.0503
   Web of Science ®Google Scholar
• Martin-Gonzalez, A., Chi-Poot, A., & Uc-Cetina, V. (2016). Usability evaluation of an augmented reality system for teaching euclidean vectors. Innovations in Education and Teaching International, 53(6), 627–636. https://doi.org/10.1080/14703297.2015.1108856
   Web of Science ®Google Scholar
• Martins, A. I., Rosa, A. F., Queirós, A., Silva, A., & Rocha, N. P. (2015). European Portuguese validation of the system usability scale (SUS). Procedia Computer Science, 67(Dsai), 293–300. https://doi.org/10.1016/j.procs.2015.09.273
   Google Scholar
• Meldrum, D., Glennon, A., Herdman, S., Murray, D., & McConn-Walsh, R. (2012). Virtual reality rehabilitation of balance: Assessment of the usability of the Nintendo Wii ® Fit Plus. Disability and Rehabilitation: Assistive Technology, 7(3), 205–210. https://doi.org/10.3109/17483107.2011.616922
   PubMedGoogle Scholar
• Meulendijk, M. C., Spruit, M. R., Drenth-van Maanen, A. C., Numans, M. E., Brinkkemper, S., Jansen, P. A. F., & Knol, W. (2015). Computerized decision support improves medication review effectiveness: An experiment evaluating the STRIP assistant’s usability. Drugs and Aging, 32(6), 495–503. https://doi.org/10.1007/s40266-015-0270-0
   PubMed Web of Science ®Google Scholar
• Minge, M., Thüring, M., & Wagner, I. (2016). Developing and validating an English version of the meCUE questionnaire for measuring user experience. Proceedings of the Human Factors and Ergonomics Society, 60(1), 2056–2060. https://doi.org/10.1177/1541931213601468
   Google Scholar
• Munsinger, B., & Quarles, J. (2019). Augmented Reality for Children in a Confirmation Task: Time, Fatigue, and Usability, 1–5. https://doi.org/10.1145/3359996.3364274
   Google Scholar
• Nawaz, A., Skjæret, N., Ystmark, K., Helbostad, J. L., Vereijken, B., & Svanæs, D. (2014). Assessing seniors’ user experience (UX) of exergames for balance training. In Proceedings of the NordiCHI 2014: The 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational (pp. 578–587). Helsinki, Finland. https://doi.org/10.1145/2639189.2639235
   Google Scholar
• Ng, A. W. Y., Lo, H. W. C., & Chan, A. H. S. (2011). Measuring the usability of safety signs: A use of system usability scale (SUS). In IMECS 2011 - International MultiConference of Engineers and Computer Scientists 2011 (Vol.2, pp. 1296–1301). Kowloon, Hong Kong.
   Google Scholar
• Nunnally, J. C. (1978a). An overview of psychological measurement. In B. B. Wolman (Ed.), Clinical diagnosis of mental disorders (pp. 97–146). Plenum Pre. https://doi.org/10.1007/978-3-540-92673-3
   Google Scholar
• Nunnally, J. C. (1978b). Psychometric theory (2 ed.). McGraw-Hill.
   Google Scholar
• Orfanou, K., Tselios, N., & Katsanos, C. (2015). Perceived usability evaluation of learning management systems: Empirical evaluation of the system usability scale. International Review of Research in Open and Distance Learning, 16(2), 227–246. https://doi.org/10.19173/irrodl.v16i2.1955
   Google Scholar
• Perlman, G. (1997). Practical usability evaluation. In CHI ’97 (pp. 369–370). https://doi.org/10.1145/223355.223726
   Google Scholar
• Peruri, A., Borchert, O., Cox, K., Hokanson, G., & Slator, B. M. (2017). Using the System usability scale in a classification learning environment. In Auer M., Guralnick D., Uhomoibhi J. (Eds.), Interactive collaborative learning. ICL 2016. Advances in intelligent systems and computing (vol. 544). Springer, Cham. https://doi.org/10.1007/978-3-319-50337-0_14
   Google Scholar
• Peterson, R. A. (2013). Meta-analysis of alpha cronbach ’ s coefficient. Journal of Consumer Research, 21(2), 381–391. https://doi.org/10.1093/bioinformatics/btr476
   Google Scholar
• Raptis, D., Tselios, N., Kjeldskov, J., & Skov, M. B. (2013). Does size matter? Investigating the impact of mobile phone screen size on users’ perceived usability, effectiveness and efficiency. MobileHCI 2013 - Proceedings of the 15th International Conference on Human-Computer Interaction with Mobile Devices and Services, 127–136. Munich, Germany. https://doi.org/10.1145/2493190.2493204
   Google Scholar
• Revythi, A., & Tselios, N. (2019). Extension of technology acceptance model by using system usability scale to assess behavioral intention to use e-learning. Education and Information Technologies, 24(4), 2341–2355. https://doi.org/10.1007/s10639-019-09869-4
   Web of Science ®Google Scholar
• Rummel, B. (2015). System Usability Scale – Jetzt auch auf Deutsch. System Usability Scale – jetzt auch auf Deutsch
   Google Scholar
• Ruoti, S., Andersen, J., Zappala, D., & Seamons, K. (2015). Why Johnny still, still can’t encrypt: Evaluating the usability of a modern PGP client. http://arxiv.org/abs/1510.08555
   Google Scholar
• Santos, M. E. C., Polvi, J., Taketomi, T., Yamamoto, G., Sandor, C., & Kato, H. (2015). Toward standard usability questionnaires for handheld augmented reality. Virtual Reality Software and Technology, 35(5), 66–75. https://doi.org/10.1109/MCG.2015.94
   Google Scholar
• Sauro, J. (2014). 4 steps to translating a questionnaire. Retrieved February 9, 2021, from https://measuringu.com/translation/
   Google Scholar
• Sauro, J. (2015). SUPR-Q: A comprehensive measure of the quality of the website user experience. Journal of Usability Studies, 10(2), 68–86. https://uxpajournal.org/supr-q-a-comprehensive-measure-of-the-quality-of-the-website-user-experience/
   Web of Science ®Google Scholar
• Sauro, J., & Lewis, J. R. (2011). When designing usability questionnaires, does it hurt to be positive? Conference on Human Factors in Computing Systems - Proceedings, 2215–2223. Vancouver, BC, Canada. https://doi.org/10.1145/1978942.1979266
   Google Scholar
• Sauro, J., & Lewis, J. R. (2012). Quantifying the user experience. Elsevier. https://doi.org/10.1016/B978-0-12-384968-7.00013-8
   Google Scholar
• Schrepp, M., Hinderks, A., & Thomaschewski, J. (2014). Applying the user experience questionnaire (UEQ) in different evaluation scenarios. In A. Marcus (Ed.), Design, user experience, and usability. Theories, methods, and tools for designing the user experience (pp. 383–392). Springer International Publishing.
   Google Scholar
• Sharfina, Z., & Santoso, H. B. (2016). An Indonesian adaptation of the system usability scale (SUS). In 2016 International Conference on Advanced Computer Science and Information Systems, ICACSIS 2016 (pp. 145–148). Malang, Indonesia. https://doi.org/10.1109/ICACSIS.2016.7872776
   Google Scholar
• South, H., Taylor, M., Dogan, H., & Jiang, N. (2017). Digitising a medical clerking system with multimodal interaction support. ICMI 2017 - Proceedings of the 19th ACM International Conference on Multimodal Interaction, 2017-Janua, 238–242. Glasgow, UK. https://doi.org/10.1145/3136755.3136758
   Google Scholar
• Tassabehji, R., & Kamala, M. A. (2012). Evaluating biometrics for online banking: The case for usability. International Journal of Information Management, 32(5), 489–494. https://doi.org/10.1016/j.ijinfomgt.2012.07.001
   Web of Science ®Google Scholar
• Tolentino, G. P., Battaglini, C., Pereira, A. C. V., De Oliveria, R. J., & De Paula, M. G. M. (2011). Usability of serious games for health. In Proceedings - 2011 3rd International Conferenceon Games and Virtual Worlds for Serious Applications, VS-Games 2011 (pp. 172–175). Athens, Greece. https://doi.org/10.1109/VS-GAMES.2011.33
   Google Scholar
• Tossell, C. C., Kortum, P., Shepard, C., Rahmati, A., & Zhong, L. (2012). An empirical analysis of smartphone personalisation: Measurement and user variability. Behaviour and Information Technology, 31(10), 995–1010. https://doi.org/10.1080/0144929X.2012.687773
   Web of Science ®Google Scholar
• Triantafyllidis, A. K., Koutkias, V. G., Chouvarda, I., & Maglaveras, N. (2014). Development and usability of a personalized sensor-based system for pervasive healthcare. In 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2014 (pp. 6623–6626). Chicago, IL, USA. https://doi.org/10.1109/EMBC.2014.6945146
   Google Scholar
• Vallerand, R. J. (1989). Vers une méthodologie de validation trans-culturelle de questionnaires psychologiques : Implications pour la recherche en langue française. Psychologie Canadienne, 30(4), 662–680. https://doi.org/10.1037/h0079856
   Web of Science ®Google Scholar
• Van De Vijver, F. J. R., & Leung, K. (2001). Personality in cultural context: Methodological issues. Journal of Personality, 69(6), 1007–1031. https://doi.org/10.1111/1467-6494.696173
   PubMed Web of Science ®Google Scholar
• Van Veenendaal, E. (1998). Questionnaire based usability testing. In Conference Proceedings European Software Quality Week, Brussels, November 1998 (pp. 1–9).
   Google Scholar
• Yang, D., Zhang, D., Chen, L., & Qu, B. (2015). NationTelescope: Monitoring and visualizing large-scale collective behavior in LBSNs. Journal of Network and Computer Applications, 55(August), 170–180. https://doi.org/10.1016/j.jnca.2015.05.010
   Google Scholar
• Yang, T., Linder, J., & Bolchini, D. (2012). DEEP: Design-oriented evaluation of perceived usability. International Journal of Human-Computer Interaction, 28(5), 308–346. https://doi.org/10.1080/10447318.2011.586320
   Web of Science ®Google Scholar