Assessing the Prevalence of Cannabis Use Through a Survey About Criminal Activity Versus One About Alcohol, Tobacco, and Other Drugs

Abstract

Background: The prevalence of cannabis use in the United Kingdom might be underestimated using the Crime Survey of England and Wales. The current study examined whether responding to questions about their cannabis use as part of a crime survey would be less likely to report that they use cannabis compared to those responding to the same questions that are part of a survey about health. Methods: Participants were randomized to be told that the items about cannabis use came from a crime survey versus from a health survey. In addition, the sample was recruited using a representative online sampling method and compared to published rates of self-reported cannabis use collected as part of the Crime Survey for England and Wales. Results: There was no significant difference (p > 0.05) in the proportion endorsing cannabis use between those told the items came from a crime survey versus a health survey. However, self-reported rates of cannabis use collected as part of the online panel (51.3% ever use; 11.9% past year; age range 18–64 years) appeared higher than those reported based on results from the Crime Survey for England and Wales (37.2% ever and 5.8% past year; age range 18–59 years). Conclusion: The current study did not find evidence that manipulating whether participants were told that the items asking about cannabis use came from a survey asking about criminal activity versus one about health had an impact on self-reported cannabis use. However, as prevalence estimates generated by the Crime Survey of England and Wales do appear to be an underestimate of actual levels of cannabis use in the United Kingdom, further research is merited on this topic.

Keywords:

• Cannabis
• prevalence
• assessment methods

1. Introduction

Many jurisdictions generate estimates of the prevalence of cannabis use in the general population as part of their ongoing monitoring of health-related behaviors. These estimates are often derived using epidemiological surveys targeting health status and health-related behaviors in the general population, or as part of health surveys specifically targeting the use of addictive substances including alcohol, tobacco, cannabis, and other drugs. England and Wales are unusual in this regard as they generate annual estimates of cannabis use employing items incorporated in a survey of the experience of criminal activity. The survey employed is the Crime Survey of England and Wales, a large annual survey conducted for the Office for National Statistics (Office for National Statistics, 2021). Using a survey asking about criminal activities to asking about cannabis use makes some sense because use of cannabis is illegal in England and Wales (unless it is prescribed for medical use). However, it may have the unintended effect of leading to an underestimate of the prevalence of cannabis use in this region, a point which has been noted in the documentation accompanying the Crime Survey of England and Wales.

It is important to have an accurate estimate of the prevalence of cannabis use because, while more research is needed to clarify the impact of cannabis use on health, it is clear that cannabis use is associated with harm (Hall et al., 2019). Accurate estimates on the prevalence of use could help in several domains. First, given that use results in harm, an accurate estimate would allow for better planning of the type and amount of treatment services needed to address this public health issue. Second, accurate estimates of prevalence are essential in research to understand the impact of cannabis on harm experienced. As an example, if a higher prevalence of people use cannabis that was previously estimated, then this means that estimates of the prevalence of cannabis users who go on to become dependent of its use is lower than anticipated. Finally, it could be argued that a different policy approach might be more effective with a harm causing behavior (in this case, cannabis use) if a small versus a large proportion of the population engages in the behavior. As one issue for consideration—is criminalization the best approach to reduce harm if a large segment of the population engages in the behavior?

Outside of the Crime Survey of England and Wales, there is limited existing literature on the prevalence of cannabis use in the UK. Ramsay and Percy, (1997) reviewed the earlier work in this area. A more recent report of cannabis prevalence employed participants recruited using an online panel survey (YouGov) (Hindocha et al., 2021). However, findings from this survey are less useful as a comparator to the findings from the Crime Surveys of England and Wales because the YouGov sample responded to a single combined item asking about frequency of cannabis use (from never to daily) rather than to the series of items employed on the Crime surveys. Thus, any difference observed could be due to the differences in the question structure rather than the context of the items (i.e., asking about participants’ use of cannabis as part of a crime survey versus as part of a YouGov ongoing panel survey).

The purpose of the current research was to evaluate whether there may be an impact on estimates of cannabis prevalence, and on stigma toward cannabis users, when participants’ were randomized to be told that the relevant items came from the ‘Crime Survey of England’ versus the ‘Health Survey of England.’ It was hypothesized that, when participants were told that the cannabis items came from a crime survey, they would be less likely to endorse that they use cannabis and would rate cannabis use as a more stigmatized activity, compared to when participants were told that the cannabis items came from a health survey. This is because being told that the items came from a crime survey might have a social desirability or demand characteristic impact on participants (McCambridge et al., 2012; Orne, 1962). Specifically, that cannabis use would be regarded as a more negative activity when associated with a crime survey than when participants were told that the items come from a health survey. Finally, the current research compared reported cannabis use from an online panel sampled to be representative of the general population to published prevalence estimates of cannabis use generated from the Crime Survey of England and Wales.

2. Methods

Participants were recruited through the Prolific website (Palan & Schitter, 2018) employing an advertisement asking for participants to complete a survey on people’s beliefs about alcohol, tobacco and other drug use. Participants were told the survey would take 5–10 min to fill out and that payment was £1.25. Prolific restricts participants to those 18 years and older. Participants who clicked on the advertisement were taken to an information sheet. Those who agreed to participate in the study after receiving the information sheet were taken to the survey.

The survey started with a series of 16 questions asking how serious a problem participants thought a variety of different societal issue were (Cunningham & Koski-Jännes, 2019; Hirschovits-Gerz et al., 2011; Holma et al., 2011). Responses to these items are reported elsewhere (Cunningham et al., in press). An attention check item was nested in the middle of these questions (I want to indicate that I have read this question by checking 10) (Godinho et al., 2016). Those who answered the attention check question incorrectly were excluded from the analyses (but were still paid for completing the survey). The survey continued by asking participants about their smoking status and the quantity and frequency of their current alcohol use (items taken from the Smoking and Alcohol Toolkit Study) (Beard et al., 2015; West, 2006). Participants were also asked about their cannabis use using items taken from the Crime Survey for England and Wales (ever use, past year use, past month use) (Office for National Statistics, 2020, 2021). Immediately before the cannabis question, participants were randomly assigned to see one of two versions of text—one saying that the cannabis items came from the Crime Survey of England and the other saying that the items came from the Health Survey of England (Crime survey versus Health survey). After completing the cannabis use items, participants were asked three items relevant to stigma toward cannabis users (most people believe that someone who uses cannabis is dangerous; is not a good person; is unreliable—response options: strongly disagree, disagree, agree, strongly agree) (Ahern et al., 2007). The survey concluded with a series of questions asking about participants’ demographic characteristics. Finally, participants were provided a brief description of the study, including the manipulation of being told that the cannabis items came from a crime or health survey (while participants were informed that not everyone would receive the same materials prior to completing the survey, they were not informed of this manipulation prior to completing the survey). Participants were then provided the option to have their data removed from the study (while still being paid).

The survey recruitment employed Prolific’s representative sampling method resulting in a sample of participants stratified to have equal numbers of participants by age groups, gender (male/female) and ethnic groups as those reported in the 2011 UK Census (Office for National Statistics, 2016). As the resulting sample appeared skewed to a younger age group than the general population, and in order to have a sample that more closely represented the general population at the time the survey was conducted (December, 2021), post-stratification weights by age group and gender (male/female) were applied using data from the Office for National Statistics estimate of the population for the United Kingdom in July of 2021 (Office for National Statistics, 2022a; Royal, 2019). For the 15 participants who did not identify as male or females, a weight of 1 was applied in order to retain these participants in the sample for analyses. Statistical tests, proportions, means, and standard deviations are presented based on weighted data. Sample sizes are reported as unweighted data.

2.1. Statistical analyses

Bivariate comparisons were employed to compare the demographic characteristics between participants randomized to be told that the cannabis use items came from the Crime Survey of England versus the Health Survey of England (Fischer’s exact tests). The same analytical approach was employed for the participant’s cannabis use items and for the items on stigma toward cannabis users. For these latter items, because six separate tests were conducted (three cannabis items and three stigma items), a Bonferroni adjustment was applied to the significance level (0.05/6 = 0.008). Finally, only visual comparisons were made between the cannabis use reported by participants on the Prolific survey and results reported on the Crime Survey of England and Wales. Further, as the Crime Survey of England and Wales restricted participation to those 18 to 59 years of age during this time period, only participants from the closest similar age range were selected from the Prolific survey (18–64 years of age) (Office for National Statistics, 2022b).

2.2. Sample size estimate

The sample size estimate was generated to test for a small impact of varying the stated source of the questions (Crime survey or Health survey) on the proportion of participants who say that they used cannabis in the past month. Specifically, the Crime Survey for England and Wales reported that, in the year ending March 2020, 7.8% of 16–59 year olds used cannabis in the past year and that 33.7% of past year cannabis users said they used cannabis in the past month (i.e., 2.6% of all participants reported using cannabis in the past month) (Office for National Statistics, 2020). A sample of 1,370 participants would allow a 0.8 power at an alpha of 0.05 to detect a 3% difference in reported past month cannabis use, assuming those told the items came from the Crime survey reported a 2.6% prevalence and those told the items came from a Health survey reported a 5.6% prevalence. Assuming that roughly 10% of the sample would be excluded because of incorrect responses on the attention check question, a recruited sample of 1,500 participants would allow for a usable sample of approximately 1,350 participants.

2.3. Ethics approval

The study received ethics approval from the REB of (King’s College London). As this was an anonymous online panel survey, participants provided consent to participate by checking that they agreed to complete the study after reading an information sheet describing the research.

3. Results

A total of 1,499 participants completed the survey. Of these 21 were removed from the data set − 5 because they did not agree to have their data included in the analyses, 2 because they reported not currently living in the UK and 14 because they did not answer the attention question check correctly. Table 1 presents the demographic characteristics of the remaining 1,478 participants, compared between those participants who were informed that they were answering cannabis items taken from a crime survey with those from a health survey (all comparisons did not reach statistical significance, p > 0.05).

Table 1. Demographic characteristics.

	Crime survey	Health survey	p
	(n = 740)	(n = 738)
% Age			0.44
18–24	10.9	10.5
25–34	17.9	17.0
35–44	16.6	16.1
45–54	15.4	17.6
55–64	17.5	14.2
65+	22.2	24.5
% Female	52.8	49.5	0.21
% Married/Common Law/Same sex partner	59.1	59.5	0.92
% Household income < UK£30,000	43.5	40.2	0.22
% Full/part time employed	60.9	60.7	1.0

3.1. Influence of describing items as coming from a health or a crime survey

Table 2 displays the percentage of participants who endorsed ever, past year, and past month cannabis use, compared between participants who were told that the items came from a Crime survey compared to a Health survey. There were no significant differences between groups (p > 0.05) with the combined self-reported prevalence of all participants (n = 1,478) being 45.1% for ever use, 9.1% for past year use, and 4.7% for past month use.

Table 2. Self-reported cannabis use and ratings of stigma toward cannabis users.

	Crime survey^a	Health survey	p
	(n = 740)	(n = 738)
% Ever used cannabis	44.1	46.1	.466
% Past year cannabis use	8.5	9.6	.472
% Past month cannabis use	4.6	4.9	.808
% Strongly agree that most people believe that someone who uses cannabis is ….
Dangerous	2.7	1.6	.156
Not a good person	3.5	2.8	.461
Unreliable	14.2	8.5	.001

^aParticipants were randomized to be told that the cannabis items came from the ‘Crime Survey of England or the Health Survey of England’.

Table 2 also displays participants’ ratings of whether most people believed that someone who uses cannabis is dangerous, not a good person, and is unreliable (three separate items—all scored as strongly agree versus all other for the analyses; please note: pattern of results remain the same when response options are not collapsed). There was no significant difference between participants told the items came from a Crime or a Health survey for the first two items (p > 0.05). For the third item, more participants strongly agreed that someone who uses cannabis is unreliable in the Crime survey group compared to the Health survey group (14.2% versus 8.5% respectively; Fischer’s exact test, p < 0.001).

3.2. Comparing to published prevalence estimates

On the Prolific survey, lifetime cannabis use was endorsed by 51.3% and past year cannabis use by 11.9% (past month 6.1%; age range restricted to 18–64). Preliminary data from the Crime Survey is now available for the year ending March 2021, including reports on lifetime and past year cannabis use (37.2% and 5.8% respectively; age range 18–59) (Office for National Statistics, 2022b).

Given that the Prolific sampling method relies on an online panel rather than a random sampling survey methodology, some estimate of the validity of the Prolific survey would strengthen confidence in the accuracy of the cannabis results reported here. To this purpose, the prevalence of current cigarette smokers was estimated to be 14.4% in December 2021 by the ongoing Smoking Toolkit Study (note: the Smoking Toolkit Study did not sample participants from all of the UK in 2021) (Fidler et al., 2011; West et al., 2022). In the Prolific survey, and using the same item to assess current smoking, the prevalence was estimated to be 13%.

4. Discussion

Participants who were informed that the cannabis items were from a Health survey did not appear more likely to say that they had used cannabis compared to those told that the items came from a Crime survey (p > 0.05). There was some limited evidence that the Health survey versus Crime survey manipulation did impact on participants’ impressions about cannabis users, with those told that the items were from a Crime survey being more likely to agree that people believe cannabis users were unreliable compared to those told the items come from a Health survey (p < 0.001). One explanation for this finding is that describing the cannabis items as coming from the Crime Survey of England might have a ‘priming’ effect, where being the association of cannabis with crime might lead participants to have more negative associations with cannabis use (Domke et al., 1998). Another possibility is that there are demand characteristics (i.e., the participant may believe that we expect them to provide a certain response) resulting from telling participants that items come from a crime survey (McCambridge et al., 2012; Orne, 1962).

The overall rates of cannabis use reported on this Prolific survey were 14% higher (ever use; 6% higher for past year use) than published estimates of cannabis use reported from the Crime Survey for England and Wales collected during the same time period (ever use: 51.3% versus 37.2%; past year: 11.9% versus 5.8%). Further, as the prevalence estimates of current smokers were relatively similar to those collected as part of the Smoking Toolkit Study, it is possible that the data on cannabis use from the Prolific survey may roughly capture the prevalence of cannabis use in the UK. No claim is made here that the Prolific survey provides an accurate estimate of cannabis prevalence. However, these results should inject caution into the assumption that the Crime Survey of England and Wales cannabis prevalence estimates can be taken as a close approximation of the cannabis use of the population of England and Wales.

Why might this difference in prevalence estimates about cannabis use occur? There is an existing evidence base examining how using different survey modalities (e.g., face-to-face versus telephone survey) can impact survey prevalence estimates (McAuliffe et al., 1998). As an example, the Crime Survey for the year ending in 2021 was completed by telephone (and with participants 18–59 years of age) whereas the previous year’s survey was self-completion module appended to a face-to-face interview (and with participants 16–59 years of age). The earlier survey (ending March 2020) reported a lower lifetime cannabis use estimate than the survey ending March 2021 (participants 16–59 years of age, face-to-face interview; lifetime cannabis use 31.1%; past year cannabis use 7.8%; past month use; 3.8%) (Office for National Statistics, 2020). Another possible explanation for the difference between the prevalence estimates in the Crime survey versus the Prolific survey is the previously mentioned one that demand characteristics may be associated with saying items come from a crime survey (McCambridge et al., 2012; Orne, 1962). However, it is relevant to note that any title for a survey (e.g., one about health) might have its own demand characteristics and further, that the information on who is commissioning the survey (e.g., the government) might also have an impact. More research is needed to determine which of these explanations (or some other explanation) is relevant here.

There were a number of limitations associated with this research. While the experimental manipulation results did not support the hypothesis that asking about cannabis use on a crime survey would result in an underestimate of prevalence rates, this may be because the manipulation was minimal. It is possible that a more substantial manipulation (e.g., provide a mockup of a full crime survey versus a health survey to participants with the cannabis items nested within the full survey) would result in a larger difference in self-reported cannabis rates. Furthermore, the use of a sample from an online panel like Prolific, while rapidly allowing the recruitment of a sample that mirrors some of the demographic characteristics of the general population, does result in survey responses from participants of which many are experienced survey takers. Replication of the possible observed differences between the Prolific survey sample and those published form the Crime Survey of England and Wales would be valuable if there are other examples of health surveys asking about cannabis use in England (Ramsay & Percy, 1997).

5. Conclusion

We test the assumption that participants responding to questions about their cannabis use as part of a crime survey will be less likely to report that they use cannabis compared to those responding to the same questions that are part of a survey about health. While the current study did not find evidence supporting this prediction, differences were observed between the prevalence estimates of cannabis use in the UK and those found in the current survey. Further research is merited to examine reasons for the possible underestimate of the population prevalence of cannabis use that may exist using the Crime Survey of England and Wales. Such research would then allow for decisions regarding the best approaches to generate accurate estimates of cannabis use in England, which is important, not only for tracking criminal activity in the country, but also to inform policy decisions regarding cannabis use and for planning health services related to this addictive behavior.

Authors’ contributions

Both authors have made an intellectual contribution to this research. (John Cunningham) is the principal investigator, with overall responsibility for the project. He conceived the study and oversaw all aspects of the project. Both authors have contributed to the manuscript drafting process, have read, and approved the final manuscript.

Acknowledgements

John Cunningham is supported by the Nat & Loretta Rothschild Chair in Addictions Treatment & Recovery Studies.

Disclosure statement

None to declare.

Data availability statement

The data used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Additional information

Funding

Funding for the study was provided as part of a (Nat & Loretta Rothschild Chair in Addictions Treatment & Recovery Studies awarded to John Cunningham).