Evaluating Representativeness of Qualitative Text Data in Identifying UX Issues

References

• Albert, R., Jeong, H., & Barabási, A. L. (1999). Internet: Diameter of the world-wide web. Nature, 401 (6749), 130–131.
   Web of Science ®Google Scholar
• Antiqueira, L., Nunes, M. D. G. V., Oliveira, Jr, O., & Costa, L. D. F. (2007). Strong correlations between text quality and complex networks features. Physica A: Statistical Mechanics and Its Applications, 373, 811–820.
   Web of Science ®Google Scholar
• Bastian, M., Heymann, S., & Jacomy, M. (2009). Gephi: An open source software for exploring and manipulating networks. Paper presented at the ICWSM, San Jose, CA.
   Google Scholar
• Biber, D. (1993). Representativeness in corpus design. Literary and Linguistic Computing, 8 (4), 243–257.
   Google Scholar
• Bonacich, P. (1972). Technique for analyzing overlapping memberships. Sociological Methodology, 4, 176–185.
   Google Scholar
• Borgatti, S. P., Everett, M. G., & Freeman, L. C. (2002). Ucinet for Windows: Software for social network analysis. Boston, MA: Analytic Technologies.
   Google Scholar
• Borgatti, S. P., Mehra, A., Brass, D. J., & Labianca, G. (2009). Network analysis in the social sciences. Science, 323 (5916), 892–895.
   PubMed Web of Science ®Google Scholar
• Bratko, I., & Šuc, D. (2003). Qualitative data mining and its applications. CIT. Journal of Computing and Information Technology, 11 (3), 145–150.
   Google Scholar
• Callon, M., Courtial, J. P., Turner, W. A., & Bauin, S. (1983). From translations to problematic networks: An introduction to co-word analysis. Social Science Information, 22 (2), 191–235.
   Web of Science ®Google Scholar
• Campbell, J. L., Quincy, C., Osserman, J., & Pedersen, O. K. (2013). Coding in-depth semistructured interviews problems of unitization and intercoder reliability and agreement. Sociological Methods & Research, 42, 294–320, 0049124113500475.
   Web of Science ®Google Scholar
• Carley, K. M. (1993). Coding choices for textual analysis: A comparison of content analysis and map analysis. Sociological Methodology, 23 (75), 75.
   Web of Science ®Google Scholar
• Carley, K. M. (1997a). Extracting team mental models through textual analysis. Journal of Organizational Behavior, 18, 533–558.
   Web of Science ®Google Scholar
• Carley, K. M. (1997b). Network text analysis: The network position of concepts. Text Analysis for the Social Sciences: Methods for Drawing Statistical Inferences from Texts and Transcripts, 4, 79–100.
   Google Scholar
• Cha, Y.-S., Kim, K., Lee, J.-Y., Lee, J., Choi, M., Jeong, M.-H., … Oh, S.-R. (2011). MAHRU-M: A mobile humanoid robot platform based on a dual-network control system and coordinated task execution. Robotics and Autonomous Systems, 59 (6), 354–366.
   Web of Science ®Google Scholar
• Chang, C. W., Lin, C. T., & Wang, L. Q. (2009). Mining the text information to optimizing the customer relationship management. Expert Systems with Applications, 36 (2), 1433–1443.
   Web of Science ®Google Scholar
• Chomsky, N. (1980). Rules and representations. Behavioral and Brain Sciences, 3 (1), 1–15.
   Web of Science ®Google Scholar
• Collins, A. M., & Quillian, M. R. (1972). Experiments on semantic memory and language comprehension. In L. W. Gregg (Ed.), Cognition in learning and memory (pp. 263). Oxford, England: John Wiley & Sons.
   Google Scholar
• Costenbader, E., & Valente, T. W. (2003). The stability of centrality measures when networks are sampled. Social Networks, 25 (4), 283–307.
   Web of Science ®Google Scholar
• Crouch, M., & McKenzie, H. (2006). The logic of small samples in interview-based qualitative research. Social Science Information, 45 (4), 483–499.
   Web of Science ®Google Scholar
• Csermely, P. (2006). Weak links: Stabilizers of complex systems from proteins to social networks. Berlin, Germany: Springer.
   Google Scholar
• Diesner, J., & Carley, K. M. (2004). Revealing social structure from texts. Causal Mapping for Research in Information Technology, 4, 81.
   Google Scholar
• Diesner, J., & Carley, K. M. (2008). Conditional random fields for entity extraction and ontological text coding. Computational and Mathematical Organization Theory, 14 (3), 248–262.
   Web of Science ®Google Scholar
• Dorogovtsev, S. N., & Mendes, J. F. (2002). Evolution of networks. Advances in Physics, 51 (4), 1079–1187.
   Web of Science ®Google Scholar
• Dumais, S. T., Furnas, G. W., Landauer, T. K., Deerwester, S., & Harshman, R. (1988). Using latent semantic analysis to improve access to textual information. Paper presented at the Proceedings of the SIGCHI conference on Human factors in computing systems, Washington, DC.
   Google Scholar
• Estes, W. K. (1982). Models of learning, memory, and choice: Selected papers. New York, NY: Praeger.
   Google Scholar
• Foltz, P. W. (1996). Latent semantic analysis for text-based research. Behavior Research Methods, Instruments, & Computers, 28 (2), 197–202.
   Google Scholar
• Freeman, L. C. (1979). Centrality in social networks conceptual clarification. Social Networks, 1 (3), 215–239.
   Web of Science ®Google Scholar
• Glaser, B. G., Strauss, A. L., & Strutzel, E. (1968). The discovery of grounded theory; strategies for qualitative research. Nursing Research, 17 (4), 364.
   Web of Science ®Google Scholar
• Granovetter, M. (1976). Network sampling: Some first steps. American Journal of Sociology, 81 (6), 1287.
   Web of Science ®Google Scholar
• Hall, P., & Martin, M. A. (1988). On bootstrap resampling and iteration. Biometrika, 75 (4), 661–671.
   Web of Science ®Google Scholar
• Hassenzahl, M., & Tractinsky, N. (2006). User experience-a research agenda. Behaviour & Information Technology, 25 (2), 91–97.
   Web of Science ®Google Scholar
• Hoser, B., Hotho, A., Jäschke, R., Schmitz, C., & Stumme, G. (2006). Semantic network analysis of ontologies. In Y. Sure, & J. Domingue (Eds.), The semantic web: Research and applications (Vol. 4011, pp. 514–529). Berlin, Heidelberg, Germany: Springer.
   Google Scholar
• Hruschka, D. J., Schwartz, D., John, D. C. S., Picone-Decaro, E., Jenkins, R. A., & Carey, J. W. (2004). Reliability in coding open-ended data: Lessons learned from HIV behavioral research. Field Methods, 16 (3), 307–331.
   Google Scholar
• Huang, A. H., & Yen, D. C. (2013). Predicting the helpfulness of online reviews:A replication. International Journal of Human-Computer Interaction, 29 (2), 129–138.
   Web of Science ®Google Scholar
• Hwang, W., & Salvendy, G. (2010). Number of people required for usability evaluation: The 10±2 rule. Communications of the ACM, 53 (5), 130–133.
   Web of Science ®Google Scholar
• ISO 9241-210. (2009). Ergonomics of human system interaction-Part 210: Human-centered design for interactive systems (formerly known as 13407). Geneva, Switzerland: International Organization for Standardization (ISO).
   Google Scholar
• Kelly, G. A. (1955). The psychology of personal constructs. Volume 1: A theory of personality. New York, NY: WW Norton and Company.
   Google Scholar
• Kendall, M. G. (1938). A new measure of rank correlation. Biometrika, 30, 81–93. Retrieved from http://www.jstor.org/stable/2332226
   Web of Science ®Google Scholar
• Kim, G. W., Lim, J., & Yun, M. (2016). Analysis of consumer value using semantic network: The comparison of hierarchical and nonhierarchical value structures. Human Factors and Ergonomics in Manufacturing & Service Industries, 26 (3), 393–407.
   Web of Science ®Google Scholar
• Kossinets, G. (2006). Effects of missing data in social networks. Social Networks, 28 (3), 247–268.
   Web of Science ®Google Scholar
• Krippendorff, K. (2004). Content analysis: An introduction to its methodology (pp. 411–433). Thousand Oaks, CA: SAGE.
   Google Scholar
• Kucera, H. (1985). Language design, computers and brains. DTIC Document, 1–21, Naval Research, Virginia.
   Google Scholar
• Kujala, S., Roto, V., Väänänen-Vainio-Mattila, K., Karapanos, E., & Sinnelä, A. (2011). UX Curve: A method for evaluating long-term user experience. Interacting with Computers, 23 (5), 473–483.
   Web of Science ®Google Scholar
• Kulkarni, S. S., Apte, U. M., & Evangelopoulos, N. E. (2014). The use of latent semantic analysis in operations management research. Decision Sciences, 45 (5), 971–994.
   Web of Science ®Google Scholar
• Kullback, S., & Leibler, R. A. (1951). On information and sufficiency. The Annals of Mathematical Statistics, 22, 79–86.
   Google Scholar
• Kurasaki, K. S. (2000). Intercoder reliability for validating conclusions drawn from open-ended interview data. Field Methods, 12 (3), 179–194.
   Google Scholar
• Lallemand, C., Gronier, G., & Koenig, V. (2015). User experience: A concept without consensus? Exploring practitioners’ perspectives through an international survey. Computers in Human Behavior, 43, 35–48.
   Web of Science ®Google Scholar
• Landauer, T. K., Foltz, P. W., & Laham, D. (1998). An introduction to latent semantic analysis. Discourse Processes, 25 (2–3), 259–284.
   Web of Science ®Google Scholar
• Law, E. L.-C., & Hvannberg, E. T. (2004). Analysis of combinatorial user effect in international usability tests. Paper presented at the Proceedings of the SIGCHI conference on Human factors in computing systems, Zurich, Switzerland.
   Google Scholar
• Law, E. L.-C., Roto, V., Hassenzahl, M., Vermeeren, A. P., & Kort, J. (2009). Understanding, scoping and defining user experience: A survey approach. Paper presented at the Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Boston, MA.
   Google Scholar
• LeCompte, M. D., & Goetz, J. P. (1982). Problems of reliability and validity in ethnographic research. Review of Educational Research, 52 (1), 31–60.
   Web of Science ®Google Scholar
• Lee, M., Rhie, Y., Kim, J., & Lim, J. (2014, October). Development of a systematic process and automation tool for semantic network analysis on natural Language. The 15th APIEMS Conference, Jeju, Korea.
   Google Scholar
• Lewis, J. R. (2001). Evaluation of procedures for adjusting problem-discovery rates estimated from small samples. International Journal of Human-Computer Interaction, 13 (4), 445–479.
   Web of Science ®Google Scholar
• Liu, B., & Zhang, L. (2012). A survey of opinion mining and sentiment analysis. In Mining text data (pp. 415–463). New York, NY: Springer.
   Google Scholar
• Lucene Korean Analyzer. (2012). Apache software foundation. Retrieved from http://cafe.naver.com/korlucene/
   Google Scholar
• Marshall, M. N. (1996). Sampling for qualitative research. Family Practice, 13 (6), 522–526.
   PubMed Web of Science ®Google Scholar
• Martin, M. K., Pfeffer, J., & Carley, K. M. (2013). Network text analysis of conceptual overlap in interviews, newspaper articles and keywords. Social Network Analysis and Mining, 3 (4), 1165–1177.
   Google Scholar
• McAuley, J., & Leskovec, J. (2013). Hidden factors and hidden topics: Understanding rating dimensions with review text. Paper presented at the Proceedings of the 7th ACM conference on Recommender systems, Hong Kong, China.
   Google Scholar
• Miles, M. B., & Huberman, A. M. (1984). Qualitative data analysis: A sourcebook of new methods. Beverly Hills, CA: SAGE Publications.
   Google Scholar
• Morse, J. M. (1994). Designing funded qualitative research. In N. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research (pp. 220–235). Thousand Oaks, CA: Sage.
   Google Scholar
• Morstatter, F., Pfeffer, J., Liu, H., & Carley, K. M. (2013). Is the sample good enough? Comparing data from Twitter’s streaming API with Twitter’s firehose. Proceedings of ICWSM, Boston, MA.
   Google Scholar
• Mudambi, S. M., & Schuff, D. (2010). What makes a helpful review? A study of customer reviews on Amazon. com. MIS Quarterly, 34 (1), 185–200.
   Web of Science ®Google Scholar
• Müller, O., Schmiedel, T., Gorbacheva, E., & vom Brocke, J. (2016). Towards a typology of business process management professionals: Identifying patterns of competences through latent semantic analysis. Enterprise Information Systems, 10 (1), 50–80.
   Web of Science ®Google Scholar
• Ngai, E. W., Xiu, L., & Chau, D. C. (2009). Application of data mining techniques in customer relationship management: A literature review and classification. Expert Systems with Applications, 36 (2), 2592–2602.
   Web of Science ®Google Scholar
• Nielsen, J. (1994). Estimating the number of subjects needed for a thinking aloud test. International Journal of Human-Computer Studies, 41 (3), 385–397.
   Web of Science ®Google Scholar
• O’Reilly, M., & Parker, N. (2012). ‘Unsatisfactory saturation’: A critical exploration of the notion of saturated sample sizes in qualitative research. Qualitative Research, 13 (2), 190–197.
   Web of Science ®Google Scholar
• Pardo, T. A. S., Antiqueira, L., Nunes, M. D. G. V., Oliveira, O. N., & Costa, L. D. F. (2006). Using complex networks for language processing: The case of summary evaluation. Paper presented at the 2006 International Conference on Communications, Circuits and Systems, Guangxi, China.
   Google Scholar
• Park, J., Han, S. H., Kim, H. K., Moon, H., & Park, J. (2015). Developing and verifying a questionnaire for evaluating user value of a mobile device. Human Factors and Ergonomics in Manufacturing & Service Industries, 25 (6), 724–739.
   Web of Science ®Google Scholar
• Popping, R. (2003). Knowledge graphs and network text analysis. Social Science Information, 42 (1), 91–106.
   Web of Science ®Google Scholar
• Quillian, M. R. (1969). The teachable language comprehender: A simulation program and theory of language. Communications of the ACM, 12 (8), 459–476.
   Web of Science ®Google Scholar
• Quinn, P. M. (2002). Qualitative research and evaluation methods. California, EU: Sage Publications Inc.
   Google Scholar
• Ramsey, C. A., & Hewitt, A. D. (2005). A methodology for assessing sample representativeness. Environmental Forensics, 6 (1), 71–75.
   Web of Science ®Google Scholar
• Rayson, P., & Garside, R. (2000). Comparing corpora using frequency profiling. Paper presented at the Proceedings of the workshop on Comparing Corpora, Hong Kong, China.
   Google Scholar
• Reynolds, T. J., & Gutman, J. (1988). Laddering theory, method, analysis, and interpretation. Journal of Advertising Research, 28 (1), 11–31.
   Web of Science ®Google Scholar
• Rousseau, R., & Zhang, Q. (1992). Zipf’s data on the frequency of Chinese words revisited. Scientometrics, 24 (2), 201–220.
   Web of Science ®Google Scholar
• Savoy, A., Guo, Y., & Salvendy, G. (2009). Effects of importance and detectability of usability problems on sample size requirements. International Journal of Human–Computer Interaction, 25 (5), 430–440.
   Web of Science ®Google Scholar
• Schmettow, M. (2012). Sample size in usability studies. Communications of the ACM, 55 (4), 64–70.
   Web of Science ®Google Scholar
• Steyvers, M., & Tenenbaum, J. B. (2005). The large‐scale structure of semantic networks: Statistical analyses and a model of semantic growth. Cognitive Science, 29 (1), 41–78.
   PubMed Web of Science ®Google Scholar
• Thomson, S. (2011). Sample size and grounded theory. Journal of Administration and Governance, 5 (1), 45–52.
   Google Scholar
• Toutanova, K., Klein, D., Manning, C., & Singer, Y. (2003). Feature-rich part-of-speech tagging with a cyclic dependency network. In Proceedings of HLT-NAACL 2003 (pp. 252–259), Edmonton, Canada.
   Google Scholar
• Valente, T. W., Coronges, K., Lakon, C., & Costenbader, E. (2008). How correlated are network centrality measures? Connections (Toronto, Ont.), 28 (1), 16–26.
   PubMedGoogle Scholar
• Visser, M., van Biljon, J., & Herselman, M. (2013). Evaluation of management information systems: A study at a further education and training college. SA Journal of Information Management, 15 (1), 8 pages.
   Google Scholar
• Wasserman, S., & Faust, K. (1994). Social network analysis: Methods and applications (Vol. 8). New York, NY: Cambridge University Press.
   Google Scholar
• Watts, D. J., & Strogatz, S. H. (1998). Collective dynamics of ‘small-world’ networks. Nature, 393 (6684), 440–442.
   PubMed Web of Science ®Google Scholar
• Wilcoxon, F. (1945). Individual comparisons by ranking methods. Biometrics Bulletin, 1, 80–83.
   Google Scholar
• Wilkins, A. J. (1971). Conjoint frequency, category size, and categorization time. Journal of Verbal Learning and Verbal Behavior, 10 (4), 382–385.
   Google Scholar
• Wright, P. C., & Monk, A. F. (1991). A cost-effective evaluation method for use by designers. International Journal of Man-Machine Studies, 35 (6), 891–912.
   Google Scholar
• Yu, C. H., Jannasch-Pennell, A., & DiGangi, S. (2011). Compatibility between text mining and qualitative research in the perspectives of grounded theory, content analysis, and reliability. The Qualitative Report, 16 (3), 730.
   Google Scholar
• Zhuang, L., Jing, F., & Zhu, X.-Y. (2006). Movie review mining and summarization. Paper presented at the Proceedings of the 15th ACM international conference on Information and knowledge management, 43–50, Virginia.
   Google Scholar
• Zipf, G. K. (1929). Relative frequency as a determinant of phonetic change. Harvard Studies in Classical Philology, 40, 1–95.
   Google Scholar