Why don’t we care more about carelessness? Understanding the causes and consequences of careless participants

ABSTRACT

Although careless respondents have wreaked havoc on research for decades, the prevalence and implications of these participants has likely increased due to many new methodological techniques currently in use. Across three studies, we examined the prevalence of careless responding in participants, several means of predicting careless respondents, and the implications of careless respondents on data quality and recruitment attempts. At the same time, we sought to examine the geographic differences of careless responding and provide psychometric evidence for using embedded questions to detect these participants. In Study 1, we found over 1/3^rd of participants showed some evidence of careless responding and that careless respondents exhibited certain personality and demographic characteristics. In particular, carelessness seemed more prevalent in Asian samples than in North American samples. In Study 2, we found nearly 1/4^th of participants showed evidence of careless responding and that conclusions based on data including versus excluding these participants differed in significant yet unpredictable ways. Finally, in Study 3, we found that nearly 2/3^rd of participants who signed up for the study did not meet advertised study requirements for participation and including these participants changed the structure of the data obtained.

KEYWORDS:

• Careless Responding
• Personality
• Individual Differences
• MTurk
• Data Quality

Disclosure statement

No potential conflict of interest was reported by the authors.

Supplementary materials

Supplemental data for this article can be accessed here.

Additional information

Funding

The College of Liberal Arts Faculty Research Fund, Rochester Institute of Technology.

Notes on contributors

Austin Lee Nichols

Austin Lee Nichols is the Director of Research for the Connection Lab and an adjunct faculty at the University of Navarra. Prior to this, he worked in various faculty positions around the world in both Psychology and Management. He received his Ph.D. in Psychology and M.S. in Management from the University of Florida. His research interests include research methodology, leadership, and impression management, and his work has been published in journals around the world and in various disciplines. He has won awards from various global organizations for his teaching, research, and service.

John E. Edlund

John E. Edlund is Associate Professor of Psychology at the Rochester Institute of Technology. He serves as the Research Director of Psi Chi: The International Honor Society in Psychology, and an associate editor of Psychological Reports. He received his doctorate from Northern Illinois University. He has won numerous awards related to teaching and mentoring and is passionate about the dissemination of psychological knowledge. His research interests are quite varied, ranging from exploring meta-research research questions, evolutionary theory being applied to interpersonal relationships, to social and legal psychologies. He has published 50 peer-reviewed articles and edited chapters, and well as editing a research methods textbook.