Evaluating Representativeness of Qualitative Text Data in Identifying UX Issues

ABSTRACT

With increasing recognition of user experience (UX), it became important to identify UX issues from customers’ opinions expressed in a form of text that is unstructured qualitative data. Since reliability of qualitative data has little been investigated, it requires much time and effort for researchers to define meanings and values from the qualitative data. Therefore, this article proposes a novel approach to measuring the stability of textual data by adopting semantic network analysis, as shown by our analysis on in-depth interview transcripts and online customer review data. Among the semantic networks generated from text, subnetworks were sampled from the original networks until the representativeness of each sample size was determined. Then, similarities between the subnetworks and the original network were calculated by applying a correlation analysis to determine whether stability occurs. The results indicated that variables of word frequency and network-level statistics affect stability of the network. The suggested method will be able to assist in identifying UX issues.

Funding

This work was supported by the National Research Foundation of Korea. Grant Number: NRF-2015R1A2A2A04007359.

Additional information

Funding

This work was supported by the National Research Foundation of Korea. Grant Number: NRF-2015R1A2A2A04007359.

Notes on contributors

Ye Lim Rhie

Ye Lim Rhie is a PhD candidate at the Department of Industrial Engineering of Seoul National University. She received BS degree in Industrial Engineering from Hongik University in 2012. Her research interests are Human–Computer Interaction and user research method.

Ji Hyoun Lim

Ji Hyoun Lim is a Human Factors Engineer at Apple Inc. She received a BS in Industrial Engineering from Seoul National University and an MS and PhD in Industrial and Operations Engineering from the University of Michigan. Her research interests are computational cognitive modeling, semantic network analysis, and human automation interaction.

Myung Hwan Yun

Myung Hwan Yun is a Professor in the Department of Industrial Engineering at Seoul National University. He received a BS and MS in industrial engineering from Seoul National University and a PhD in industrial and manufacturing engineering from Penn State University. His research interests include Human Factors and Affective Product Design.