Bots and nots: safeguarding online survey research with underrepresented and diverse populations

ABSTRACT

COVID-19 has intensified the use of internet-mediated research for reaching historically underrepresented minorities (URMs) and other diverse groups. Preventing and detecting responses from automated accounts ‘bots’ and those who misrepresent themselves is one challenge in utilising online approaches. Through internet-mediated methods, interested LGBTQ+ and non-LGBTQ+ couples facing advanced cancer completed an interest form via REDCap®. Eligible participants received a direct link to electronic consent and surveys in REDCap®. Once responses to the interest form (N = 619) were received, the study PI: 1) assessed participants’ entries and non-response survey data (time of completion, rate of recruitment, etc.), 2) temporarily postponed recruitment, 3) sent eligibility questionnaires, consent documents, and validated surveys to N = 10 couples and scrutinised these data for suspicious patterns or indications of untrustworthy data, 4) responded to potential participants via email, and 5) implemented additional strategies for detecting and preventing untrustworthy survey responses. Investigators must consider multi-step eligibility screening processes to detect and prevent the collection of untrustworthy data. Investigators’ reliance on internet-mediated approaches for conducting research with diverse, hard-to-reach populations increases the importance of addressing threats to data validity. Ultimately, safeguarding internet-mediated research supports research accessibility and inclusion for URMs while also protecting participant data integrity.

KEYWORDS:

• Internet-mediated research
• data collection
• surveys and questionnaires
• minority groups
• lgbtq persons

Acknowledgments

The authors would like to acknowledge the National Center for Advancing Translational Sciences and the National Institute of Nursing Research of the National Institutes of Health for supporting this research.

Disclosure of potential conflicts of interest

There are no known conflicts of interest associated with this publication. This is an original manuscript and it not being considered elsewhere for publication.

Additional information

Funding

This work was supported by the National Center for Advancing Translational Sciences [UL1TR002538]; National Institute of Nursing Research [T32NR013456].

Notes on contributors

Sara Bybee

Sara Bybee, LCSW is a Ph.D. Candidate at the University of Utah College of Nursing. She earned a Master’s degree in Social Work from Portland State University in 2012 and Bachelor’s degrees in Community Health and Spanish from Tufts University in 2008. She has been employed as an oncology social worker at Huntsman Cancer Institute for the past five years. During her doctoral studies, Sara earned Gerontology Interdisciplinary Program and Nursing Education Graduate Certificates. She conducts research with interdisciplinary teams studying family and cancer caregiving, hospice visits, hospice team attitudes towards LGBTQ families, and dyadic interactions among cancer patients and their partner caregivers. She has presented research at: The Society of Behavioral Medicine, Social Work in Hospice and Palliative Care Network, the Gerontological Society of America, and the Western Institute of Nursing conferences. Sara is a Jonas Nurse Scholar alumni and is currently a predoctoral fellow in Interdisciplinary Training in Cancer, Caregiving, and End-of-Life Care through the National Institutes of Health. Sara serves as a co-convener of the Rainbow Research special interest group of the Gerontological Society of America, and as Utah’s Advocacy Leader with the Graduate Nursing Student Academy.

Kristin Cloyes

Kristin Cloyes is an associate professor at the University of Utah College of Nursing. She investigates family and informal caregiving in communities with flexible and adaptive family and kinship structures that have been underserved or marginalized by health care and research. She is especially interested in how networks of family, friends, and close others provide social support and caregiving for members with chronic illness and at end of life. Since 2014, her program of research has focused on LGBTQ+ individuals with chronic and life‐limiting illness and their caregivers. She is leading an NIH Administrative Supplement on Dr. Ellington’s R01 (NR016249) investigating support and communication needs for LGBTQ hospice family, and is the PI on a multi-site project utilizing mEMA with LGBTQ and AYA advanced cancer patients and their primary caregivers to assess patterns of support interactions within their social networks. Recently, she led a mixed-methods study to investigate how distal sources of minority stress─ including interactions with health care providers—affect chronic illness, pain, and mental health outcomes for older adults in these groups.

Brian Baucom

Brian Baucom is an Associate Professor of Psychology at the University of Utah. His research integrates trait questionnaires, daily diary, psychophysiological, observational, and sensor-based measurement of behavior and emotion using data acquired in the laboratory as well as outside of the laboratory during daily life. He has extensive expertise in collecting, processing, and dyadic observational and self-report data. He is currently the m-PI of a $2,355,470 project funded by the Department of Defense that develops signal processing technologies and machine learning algorithms for risk for suicide based on observationally coded dyadic behavior and individual performance on cognitive performance tasks. He is also currently a co-I on eight different federally funded and foundation grants five of which involve the study of dyadic processes linked to mental and physical health outcomes including four different grants that examine close relationships. His role in each of these projects includes both substantive and statistical/methodological roles that draw on his substantial quantitative expertise in complex dyadic analysis. In addition to these eight current grants on which he is a co-I, he serves as a consultant on three more federally or internationally funded grants that examine dyadic processes and has also been a co-I or consultant on eight completed federally or internationally funded grants, six of which focused on dyadic research.

Katherine Supiano

Katherine Supiano is an Associate Professor and the Director of Caring Connections: A Hope and Comfort in Grief Program, the bereavement care program of the University of Utah serving the intermountain west. She earned her PhD from the University of Utah College of Social Work in 2012. She has been a practicing psychotherapist in geriatrics, palliative care and grief therapy for over 35 years. She is a Fellow in the Gerontological Society of America, a Fellow of Thanatology-Association of Death Education and Counseling, and a member of the Social Work Hospice and Palliative Care Network Board of Directors. Her research focuses on developing, implementing and evaluating training of clinicians to address the Grief of Overdose Death and the Grief of Suicide Death (Utah Division of Substance Abuse and Mental Health). She is also working towards evaluating a prevention model for dementia caregivers at risk for complicated grief, funded by the Alzheimer’s Association. She has received an Innovations in Care grant from the Rita and Alex Hillman Foundation to conduct a feasibility study of Complicated Grief Group Therapy and has received 1U4U Innovation grant from the University of Utah.

Kathi Mooney

Kathi Mooney is a Distinguished Professor at the University of Utah College of Nursing and holds the Louis S. Peery and Janet B. Peery Presidential Endowed Chair in Nursing. She is an investigator and co-leader of the Cancer Control and Population Sciences Program at the Huntsman Cancer Institute. Her research focuses on cancer patient symptom remote monitoring and management, technology-aided interventions, cancer family caregivers, improving supportive care outcomes for cancer patients in rural/frontier communities and models of cancer care delivery. Dr. Mooney has been continuously funded through National Cancer Institute grants for the past 20 years. She has published numerous book chapters and journal articles and is a frequent speaker on topics related to cancer symptom management, quality cancer care, new delivery models, technology-aided interventions/telehealth, and creativity and innovation in science.

Lee Ellington

Lee Ellington is a Professor at the University of Utah College of Nursing, a licensed clinical psychologist, and a Huntsman Cancer Institute Investigator. She holds the Robert S. and Beth M. Carter Endowed Chair. Her interdisciplinary program of research focuses on the impact of interpersonal health communication on adjustment, decision-making, and health. Dr. Ellington has had 20 years of continuous funding from NIH and the American Cancer Society, and for the last decade her team has focused their research on hospice care of families facing life-limiting cancer. She was a Project Leader on a Program Project Grant (5P01CA138317-06; PI Mooney) on hospice family caregivers of cancer patients and nurse communication. She is currently the PI of a multi-site R01 in partnership with the Palliative Care Research Cooperative (NR016249) assessing the responsiveness of hospice care team members to the daily needs of cancer family caregivers. She is also the PI of an Administrative Supplement, led by colleague Dr. Kristin Cloyes, to examine the experiences of sexual and gender minority hospice family caregivers. Additionally, Dr. Ellington co-directs a T32 in Interdisciplinary Training Program in Cancer, Caregiving, and End-of-Life Care (T32 NR013456; mPIs Ellington & Mooney). Dr. Ellington was recently awarded a 5-year K07 Academic Leadership Award—The Family Caregiving Research Collaborative (K07 AG068185).