A Critical Assessment of Bias in Survey Studies Using Location-Based Sampling to Recruit Patrons in Bars

Abstract

Location-based sampling is a method to obtain samples of people within ecological contexts relevant to specific public health outcomes. Random selection increases generalizability; however, in some circumstances (such as surveying bar patrons), recruitment conditions increase risks of sample bias. We attempted to recruit representative samples of bars and patrons in six California cities, but low response rates precluded meaningful analysis. A systematic review of 24 similar studies revealed that none addressed the key shortcomings of our study. We recommend steps to improve studies that use location-based sampling: (i) purposively sample places of interest, (ii) use recruitment strategies appropriate to the environment, and (iii) provide full information on response rates at all levels of sampling.

Keywords :

• selection bias
• refusal bias
• systematic review
• bars
• survey
• location-based sampling

THE AUTHORS

Christopher Morrison is an associate research scientist at the Prevention Research Center (Pacific Institute for Research and Evaluation, Oakland, California). He holds an MPH in international health and is a PhD candidate at the Department of Epidemiology and Preventive Medicine at Monash University (Melbourne, Australia). His research primarily examines geospatial relationships between alcohol markets and alcohol-related problems.

Juliet P. Lee is a senior research scientist at the Prevention Research Center of the Pacific Institute for Research and Evaluation. She holds a PhD in cultural anthropology and an MA in Asian Studies. Her interests in drug, alcohol, and tobacco studies include the social construction of drugs and drug use; critique of methods; and ethnographic approaches to alcohol, tobacco, and drug policies. She has served as principal or co-investigator on numerous research projects utilizing mixed methods designs and community-partnered approaches with funding from the National Institutes of Health and the University of California.

Paul J. Gruenewald, PhD, is currently Scientific Director of the Prevention Research Center. His research interests focus upon studies of the social, economic, and physical availability of alcohol, alcohol use, and alcohol-related problems. Additional foci of his work include mathematical and statistical models of alcohol use and related problems, the development of evaluation methodologies appropriate to community-based evaluations of preventive interventions, and the environmental prevention of violence. He also directs the Spatial Systems Group, a coordinating center for work using Geographic Information Systems, Spatial Statistical Systems, and Spatial Dynamic Models. He has been a principal or co-investigator on 20 funded research projects. Dr. Gruenewald is currently principal investigator on three research projects funded by the National Institute on Alcohol Abuse and Alcoholism (NIAAA). In honor of his research achievements, Dr. Gruenewald received a Merit Award from the NIAAA to support continued studies of alcohol outlets and violence.

Miesha Marzell, PhD, is an assistant professor in the Department of Community and Behavioral Health, College of Public Health at the University of Iowa. Her broad research interests are the etiology and prevention of alcohol and drug misuse among at-risk emerging adult and racial/ethnic minority populations. Her specific interests include the study of environmental and policy factors that can influence these behaviors. Dr. Marzell also studies the relationship between sports participation and substance abuse to inform prevention efforts.

GLOSSARY

• Cooperation rate: The number of completed and partially assessed cases divided by the total number of cases approached for inclusion within locations.

• Location-based sampling: A method of recruiting respondents into a survey in which eligibility is based on their physical presence at a location of interest.

• Response rate: The number of cases that participated in the survey divided by the total number of eligible sample units. In location-based sampling when access to people at locations requires consent of the location, response rates should be separately assessed for locations and sample units.