Is some data better than no data at all? Evaluating the utility of secondary needs assessment data

Abstract

Needs assessments in substance abuse prevention often rely on secondary data measures of consumption and consequences to determine what population subgroup and geographic areas should receive a portion of limited resources. Although these secondary data measures have some benefits (e.g. large sample sizes, lack of survey response biases and cost), statistical tests on these data may not actually answer our underlying research questions. The present study sought to examine the reliability and validity of a set of alcohol consumption and consequence proxy measures as predictors of actual criterion data on youth alcohol use collected using a sampling design. The results of this study suggest that measures of more excessive alcohol use or problem behaviour occurrence are more likely to be related to criterion measures of youth alcohol use, as these secondary data tend to be more time invariant. Furthermore, the measures of less-excessive alcohol use or problem behaviour occurrence, while more normally distributed, tend to be more time variant and will typically lead policy makers to incorrect needs assessment allocation decisions. These findings suggest that policy makers should focus on secondary data measures that reflect extreme alcohol use problems, such that geographic areas that are most in need of help receive limited resources.