The reliability of naturalistic observations of social, physical and economic environments of bars

Abstract

Drinking in bars contributes to numerous public health problems, including violence and motor vehicle crashes. In order to formulate effective preventive interventions, it is essential to identify which specific features of bar environments are related to increased risks. Unobtrusive ethnographic observations are one approach that has been used to characterize these features; however, no studies have assessed reliability in a representative sample of bars. We performed brief scouting assessments in all 165 bars in six purposively selected California cities, followed by unobtrusive observations from a subsequent representative sample of 97 bars which were located in low- and high-bar density areas of the cities. Inter-rater reliability between two independent observers assessed individual item reliability, and principal components analyses assessed the reliability of a series of scales describing the physical, social, and economic characteristics of the bars. For the scouting assessment, items exhibited at least moderate reliability (κ or r ≥ 0.40). For the unobtrusive observations, items assessing physical and economic environments (e.g. pool table present, κ = 0.90; index beer cost, r = 0.82) had moderate to outstanding reliability (κ or r > 0.80). Items describing the social environment generally had poorer reliability, though group aspects (e.g. patron count, r = 0.78; patron circulation, r = 0.64) had better reliability than individual behaviors (e.g. derogatory speech, κ = 0.12). Scales constructed from specific sets of items exhibited modest reliability. The individual metrics and principal components we present will enable future studies seeking to disaggregate relationships between bar characteristics and public health problems.

Keywords:

• Bar
• drinking
• ethnography
• quantitative
• reliability
• survey

Acknowledgements

We wish to acknowledge the valuable contributions of Andy Steinburg, Neil O’Callaghan, Tim Cochran, Zev Hardman, Craig Wingate, Dayna Fisher, Phoenix Jackson, Scott Martin, and Rachelle Annechino to data collection for this study.

Disclosure statement

The authors report no conflicts of interest. The authors alone are responsible for the content and writing of this article.

Funding information

This research was supported by National Institute of Alcohol Abuse and Alcoholism (NIAAA) Grant No. R01-AA019773, and NIAAA Center Grant No. P60-AA006282.