Psychometric Properties of the Spanish Full and Short Forms of the System Usability Scale (SUS): Detecting the Effect of Negatively Worded Items

Abstract

This study examined the SUS’s psychometric properties with the Spanish population considering the plausible method effects associated with negatively worded items. A short form consisting of either direct or reversed items was also examined. Participants were 1321 Spaniards who completed the SUS. Confirmatory analyses showed that the SUS was a valid measure with a one-factor structure when method errors associated with negatively worded items were considered (CFI = .932, TLI = .898; RMSEA = .055, CI 90% = 0.047, 0.062), and shown evidence of reliability (Cronbach’s alpha = .76). The short version with only positively worded items also showed to be a valid (CFI = .973, TLI = .946; RMSEA = .057, CI 90% = .041, .075) and reliable measure (Cronbach alpha = .77) to assess usability. This is the first study to clarify the effect of negatively worded items in the structure of the SUS and propose a short version of the SUS to be used with Spaniards when a brief version is preferred to test usability.

Keywords:

• System Usability Scale
• validity
• reliability
• method effects
• Spanish
• psychometric properties

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This study was funded in part by Consellería de Innovación, Universidades, Ciencia y Sociedad Digital under the grant “Ayudas urgentes para proyectos de investigación, desarrollo tecnológico e innovación (I + D + i) por la covid19” [GVA-COVID19/2021/074]; and by MCIN/AEI/10.13039/501100011033 and ERDF under the grant [RTI2018-098349-B-I00].

Notes on contributors

Diana Castilla

Diana Castilla is a psychologist specialized in the development of eHealth applications. Her main field is Cognitive Ergonomics, with a special interest in Usability and User Experience with ICTs. She leads the research group “SAEM-TIC. Mental Health Technologies” at University of Valencia (Spain).

Irene Jaen

Irene Jaén is researcher at the Universitat Jaume I (Spain). Her main line of research is emotion regulation, with a special interest in the application of Information and Communication Technologies to the field of health and clinical psychology.

Carlos Suso-Ribera

Carlos Suso-Ribera is professor in the Universitat Jaume I (Spain). His main line of research is chronic pain and new technologies, with special interest in ecological momentary interventions and machine learning.

Gemma Garcia-Soriano

Gemma Garcia-Soriano is Professor at University of Valencia (Spain), and senior researcher of I’TOC research group at University of Valencia. Her main field of work is focused on mental health applications and Obsessive-compulsive disorder.

Irene Zaragoza

Irene Zaragoza is engineer in Computer Science with interest in the development of different eHealth applications. She is a senior researcher of “SAEM-TIC. Mental Health Technologies” group at University of Valencia (Spain).

Juani Breton-Lopez

Juani Breton-Lopez is professor at Universitat Jaume I (Spain). Her research interests include health and clinical psychology, and the application of Information and Communication Technologies to Psychology. She leads the Emotional Disorders Clinic at UJI, which offers psychological treatments to the community and training for clinical psychologists.

Adriana Mira

Adriana Mira is professor at University of Valencia (Spain), and researcher in Labpsitec. Her research interests include health and clinical psychology and the application of Information and Communication Technologies to Psychology, especially in emotional disorders.

Amanda Diaz-Garcia

Amanda Díaz-García is professor at University of Zaragoza (Spain), and researcher of IPES research group. Her research interests include health and clinical psychology, and intervention of emotional disorders through the Unified Protocol and the application of Information and Communication Technologies to Psychology

Azucena Garcia-Palacios

Azucena Garcia-Palacios is a clinical psychologist with an interest in health and clinical applications based on Information and Communication Technologies; she is professor in Psychopathology and director of Labpsitec, Jaume I University (Spain).