The Effect of COVID-19 Vaccinations on Self-Reported Depression and Anxiety During February 2021

Abstract

Using the COVID-19 Trends and Impact Survey, we estimate the average effect of COVID-19 vaccinations on self-reported feelings of depression and anxiety, isolation, and worries about health among vaccine-accepting respondents in February 2021, and find 3.7, 3.3, and 4.3 percentage point reductions in the probability of each outcome, respectively, with particularly large reductions among respondents aged 18 and 24 years old. We show that interventions targeting social isolation account for 39.1% of the total effect of COVID-19 vaccinations on depression, while interventions targeting worries about health account for 8.3%. This suggests that social isolation is a stronger mediator of the effect of COVID-19 vaccinations on depression than worries about health. We caution that these causal interpretations rely on strong assumptions. Supplementary materials for this article are available online.

Keywords:

• COVID-19
• Mental health
• Vaccinations

Supplementary Materials

Appendices: this supplement consists of appendices containing additional summary statistics; details about the causal estimands, identifying assumptions, and estimation strategy; additional results; details on the sensitivity analyses; and robustness checks.

Acknowledgments

The authors gratefully acknowledge invaluable advice from discussions with Vinni Bhatia, Nate Breg, David Choi, Riccardo Fogliato, Joel Greenhouse, Edward Kennedy, Samantha Patel, Alex Reinhart, and Robin Mejia.

Notes

1 This period encompasses both CTIS waves 7 and 8. Wave 8 was deployed on February 8, 2021, and incorporated changes to some of the questions we use as covariates. These include the addition of categories to the chronic health conditions and occupation questions, and changing a question on the previous receipt of flu vaccine to be defined from July 2020 instead of June 2020.

2 We recode the response “I don’t know” to be grouped with people who indicated they did not already receive a COVID-19 vaccine.

3 The CTIS questions specifically are “In the past five days, how often have you felt isolated from others?” and “How worried are you that you or someone in your immediate family might become seriously ill from COVID-19 (coronavirus disease)?”

4 Similar patterns were observed in other surveys: for example, an Economist/YouGov poll showed similar age patterns with respect to worries about contracting COVID-19. Clair et al. (2021) also found higher levels of social isolation among younger individuals.

5 See, e.g., https://www.nytimes.com/interactive/2021/us/covid-cases.html

6 As has been observed elsewhere, the CTIS over-represents vaccinated individuals compared to the U.S. population (Salomon et al. 2021). This does not necessarily threaten the internal validity of our analysis (see Section 4.3), though may threaten our ability to generalize our results.

7 This expression slightly simplifies of our true targeted estimand. As we explain in Section 3, we also remove those with missing FIPS code information and those who live in Alaska.

8 We caution that recent results from Miles (2022) casts doubt on mechanistic interpretations of interventional effects.

9 The conditions in the DAG are necessary, but not sufficient for our identification result (see Appendix 3, supplementary materials). For example, the DAG implies that $(R,S)\perp (Y,M_1,M_2)|(X,A,V=1)$, while we instead invoke Assumption 3. Identification under alternative conditions is possible.

10 This is similar to the “honest” tree-based estimation approach proposed by Athey and Imbens (2016), and is an example of a DR-learner, discussed in Kennedy (2020)

11 As noted in Appendix 3, supplementary materials, we may interpret these results as bounding the effect of the pandemic on depression. For example, under some assumptions we can say that the pandemic increased depression by at least 3.7 percentage points on average, and that the percentage of depressed respondents was at least 24% higher on average (100*($\frac{1}{1-.19}-1$)) than it would have been absent the pandemic among respondents in our analytic sample.

12 Our estimates are somewhat large in magnitude among respondents who do not provide demographic information. For example, among those who did not respond about their age, we estimate a –5.8 (–7.2, –4.3) percentage point reduction in depression and a –7.3 (–8.9, –5.6) reduction in worries about health. Because nonresponse is highly correlated across questions, we observe similar magnitude estimates for other nonresponse categories, though the point estimates for isolation tend to be comparable to other groups.

13 We present estimates using our XGBoost models: our GLM estimates do not include interactions between the covariates, and are therefore unlikely to correctly capture this heterogeneity.

14 We display significance levels associated with the pairwise t-tests in Appendix 5.2, supplementary materials.

15 The differences in the estimated effects are both statistically significant at the $\alpha =0.05$ level.

16 We excluded Asian and Pacific Islander and Other racial categories as the estimates are very imprecise.

17 The results using the XGBoost models are virtually identical.

18 These values reflect cases where the observed covariates are assumed to deconfound the relationship on the observed sample. This does not account for scenarios where bias is induced by selection on the observed or potential outcomes.

19 As a more formal analysis, in Appendix 6, supplementary materials we extend the previous sensitivity analysis to evaluate how our effects might generalize across a population with the same covariate distribution as our sample but include entirely cases where RS = 0. This analysis suggests that our ability to generalize our estimates may be weak; however, it is also very conservative.

20 To see this, consider the model:

$\mathbb{E}[S({\mathbf{A}}_{c-i})|x]={\delta }_{0x}+{\delta }_{1x}{A}_{ci}$ (11)

Within each covariate stratum the bias of our target quantity relative to our target is equal to ${\beta }_{2x}({\delta }_{1x}-1)$. The sample average of this bias across all covariate strata is greater than zero: ${\beta }_{2x}<0$ for all x by assumption, and ${\delta }_{1x}\le 1$ by the definition of $S({\mathbf{A}}_{c-i})$. As long as vaccinations aren’t perfectly correlated within every community, there exists some $x\prime$ in our sample where ${\delta }_{1x\prime }<1$, implying positive bias.

If vaccinations were independently assigned, ${\delta }_{1x}=0,{\tilde{\beta }}_{1x}={\beta }_{1x}$, and $\tilde{\vartheta }$ would correctly capture the direct effects of vaccination on the outcomes (but not the total direct and indirect effects). The sign of the bias of the direct effect is unclear and depends on ${\delta }_{1x}$.

21 Agrawal et al. (2021) attempt to estimate spillover effects by examining whether increases in community vaccination rates improve depression and anxiety symptoms among the unvaccinated population. They are unable to find evidence that this is the case, providing some empirical evidence that we may need not worry about SUTVA violations affecting our estimates.

22 Perez-Arce et al. (2021) speculate that in addition to worries about health and social isolation, “different work opportunities” is a possible pathway through which COVID-19 vaccines may affect depression. While this pathway is related to household finances, worries about household finances could remain the same while one’s employment status may plausibly differ. More broadly our analysis rules out any pathways via employment changes. Limiting our analysis to February again mitigates potential bias if we think people are unlikely to change their employment within a short-time frame after being vaccinated.