Preliminary Psychometric Evaluation of the Hamilton Depression Rating Scale in Methamphetamine Dependence

ABSTRACT

Objective: The purpose of this study was to test the initial psychometric properties of the 17-item Hamilton Depression Rating Scale (HAM-D) in individuals with and without major depressive disorder who use methamphetamine. We used data from two completed studies and two ongoing clinical trials. The HAM-D has well established reliability and validity in a variety of populations. However, there are no published reports of reliability and validity of the HAM-D in a methamphetamine-using population. Methods: HAM-D and depression status data were extracted from four separate studies for this psychometric assessment. Using these data, we evaluated three measures of construct validity: internal consistency, contrasted group validity, and factorial validity. Results: We found potential concerns with the construct validity of the HAM-D in users of methamphetamine. Intercorrelations between items were primarily less than 0.20 and the Cronbach's alpha value in this sample was 0.58, indicating potential issues with internal consistency. The results of two-sample t-tests suggest concerns with contrasted group validity, as no significant difference in average scores were found for nine items. Consistent with previous studies, a principal component analysis indicates that the HAM-D is multidimensional. Conclusions: The 17-item HAM-D might not reliably and validly measure depression severity in a methamphetamine-using population. Given our small sample, additional research is needed, though, to further test the psychometric properties of the HAM-D in individuals who use methamphetamine.

KEYWORDS:

• Methamphetamine
• Hamilton Depression Rating Scale
• psychometric evaluation
• depression severity

Introduction

Methamphetamine use disorders and depression are highly comorbid. It is well documented that short- and long-term methamphetamine use is associated with depression (DiMiceli, Sherman, Aramrattana, Sirirojn, & Celentano, 2016; Kalechstein et al., 2000; Nakama et al., 2008; Semple, Patterson, & Rant, 2009; Shoptaw, Peck, Redbackm, & Rotheram-Fuller, 2003) and that individuals with depression may self-medicate with methamphetamine (Hellem, Lundberg, & Renshaw, 2015; McKetin, Lubman, Lee, Ross, & Slade, 2011; Semple, Zians, Strathdee, & Patterson, 2007). The relationship between the symptoms of depression and methamphetamine intoxication and withdrawal complicates diagnosing major depressive disorder (MDD; Langås, Malt, & Opjordsmoen, 2011) and measuring depression severity in methamphetamine users due to overlap in symptoms of depression and intoxicating effects of methamphetamine. Regardless, a diagnosis of depression among individuals who use methamphetamine confounds treatment outcomes (Stein et al., 2004; Glasner-Edwards, Mooney, Marinelli-Casey, Hillhouse, Ang, & Rawson, 2010).

The 17-item HAM-D, developed in 1960, was originally administered to inpatients with a diagnosed depressive disorder (Hamilton, 1960). The items are rated on a three- or five-point scale representing the severity for each item over the prior seven days. Since its development, the HAM-D has been estimated to be the most frequently used measure of depression severity in clinical trials (Santor, Gregus, & Welch, 2006) and is often used as a point of reference for the development of new depression severity scales (Bagby, Ryder, Schuller, & Marshall, 2004).

To our knowledge, there are no published psychometric reports of the HAM-D in individuals who use methamphetamine. Therefore, the purpose of this study was to evaluate the psychometric properties of the 17-item HAM-D in adults who use methamphetamine with and without a diagnosis of MDD using data from two completed studies and two ongoing clinical trials.

Methods

Participants

This study took place at the University of Utah and Montana State University. Data were collected at screening visits from four separate studies (see Sung et al., 2013; Hellem et al., 2015; ClinicalTrials.gov ID NCT02568878 and NCT02192931). Three studies required participants to meet Diagnostic and Statistical Manual (DSM)-IV criteria (American Psychiatric Association, 2000) for current methamphetamine abuse or dependence (Sung et al., 2013; ClinicalTrials.gov ID NCT02568878 and NCT02192931) and one for methamphetamine abuse or dependence within the last 12 months (Hellem et al., 2015). Further, three studies (Hellem et al., 2015; ClinicalTrials.gov ID NCT02568878 and NCT02192931) required participants to meet DSM-IV criteria for MDD with a screening HAM-D score of 15 (Hellem et al., 2015) or 16 (ClinicalTrials.gov ID NCT02568878 and NCT02192931). Participants from these studies meeting criteria for methamphetamine abuse or dependence were included in our analysis. We extracted data from 102 participants (n = 58 with MDD, n = 21 without MDD, and n = 23 with unknown MDD status). Institutional review boards (IRBs) at the University of Utah and Montana State University issued approval for the current study.

Measures

When screened for study eligibility, the Structured Clinical Interview for DSM-IV Disorders (SCID-IV; Sung et al., 2013; Hellem et al., 2015) or the Psychiatric Research Instrument for Substance and Mental Disorders (PRISM; ClinicalTrials.gov ID NCT02568878 and NCT02192931) and the 17-item HAM-D were administered. Data on demographics, HAM-D, SCID-IV, and PRISM diagnosis (MDD or no MDD) were used in the current study.

Procedures

Trained study personnel administered the SCID-IV (Sung et al., 2013; Hellem et al., 2015) or PRISM (ClinicalTrials.gov ID NCT02568878 and NCT02192931) after written informed consent was obtained. Participants meeting SCID-IV or PRISM criteria for study inclusion completed the remaining screening procedures, including the HAM-D. A waiver of informed consent was issued by both IRBs for the current analysis, as data were collected retrospectively.

Statistical analysis

Our statistical analysis evaluated the construct validity, that is, how well an instrument measures what it is purporting to measure, of the HAM-D in adults with methamphetamine abuse or dependence. Summary statistics for demographics and HAM-D scores were calculated. Three measures of construct validity were evaluated. Internal consistency was evaluated using intercorrelations and Cronbach's alpha, contrasted group validity was assessed using two-sample t-tests, and factorial validity was examined using factor analysis. All statistical analysis was conducted in SAS 9.4.

Results

Descriptive statistics

Table 1 provides summary statistics for demographics and total HAM-D scores by depression status. Table 2 contains estimated intercorrelations between the items and total score, and Table 3 presents summary statistics for each item by depression status. Table 1 indicates that average total scores differ between the MDD and no-MDD groups, with the MDD group reporting a higher average HAM-D score. Further, our sample consisted primarily of women (87.9% in MDD group and 54.4% in non-MDD group) and individuals with current methamphetamine dependence (84.5% in MDD group and 95.2% in non-MDD group). Tables 2 and 3 are discussed in the following sections.

Table 1. Subject characteristics.

	Major Depressive Disorder	No Major Depressive Disorder	Unknown Depression Status
Participant demographics
Age, years (mean, SD)	36.2 (9.1)	30.1 (6.3)	35.2 (6.6)
Sex (%)
Female	87.9	54.4	87
Race (%)
American Indian	4 (6.9)	1 (4.8)	—
White	51 (87.9)	18 (85.7)	22 (95.7)
Black or African American	—	1 (4.8)	—
More than two races	2 (3.4)	—	1 (4.3)
Participant declined	1 (4.8)	1 (4.8)	—
Ethnicity (%)
Hispanic or Latino	3 (5.2)	3 (14.3)	5 (21.7)
Not Hispanic or Latino	55 (94.8)	18 (85.7)	18 (78.3)
Years of education (mean, SD)	12.9 (2.0)	12.7 (1.6)	12.3 (1.8)
Onset of methamphetamine use, age in years (mean, SD)	21.2 (7.5)	20.6 (5.4)	21.6 (8.2)
Methamphetamine dependence current (%)	84.5	95.2	87.5
Methamphetamine dependence lifetime (%)	98.3	100	30.4
Methamphetamine abuse current (%)	—	—	4.3
Methamphetamine abuse lifetime (%)	1.7	—	4.3
17-item Hamilton Depression Rating Scale score (mean, SD)	17.5 (4.3)	9.4 (3.9)	15.3 (5.9)

Note. SD = standard deviation.

Table 2. Estimated intercorrelations between items.

	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	Total
1	1.00
2	0.27	1.00
3	0.27	0.14	1.00
4	0.00	0.02	0.16	1.00
5	0.08	0.24	0.13	0.29	1.00
6	0.01	0.03	0.23	0.05	0.35	1.00
7	0.38	0.22	0.32	0.01	0.14	0.01	1.00
8	−0.03	−0.11	−0.06	−0.06	0.01	0.03	0.28	1.0
9	0.06	−0.03	−0.22	−0.11	0.06	−0.16	−0.12	0.18	1.00
10	0.23	0.36	0.13	0.00	0.02	0.12	0.12	0.14	0.12	1.00
11	0.26	0.27	0.06	0.03	0.26	0.17	0.00	−0.19	0.10	0.39	1.00
12	0.14	0.07	−0.08	0.11	0.06	0.14	0.08	−0.11	−0.10	0.25	0.43	1.00
13	0.26	0.33	0.03	0.05	0.21	−0.06	0.32	0.09	0.06	0.35	0.24	0.17	1.00
14	−0.09	−0.09	−0.01	−0.06	−0.08	−0.02	0.15	0.10	0.05	0.05	0.02	0.04	0.15	1.00
15	0.29	0.08	0.16	0.24	0.13	0.03	0.17	−0.01	0.06	0.21	0.16	−0.04	0.34	0.12	1.00
16	−0.06	−0.05	0.00	−0.07	0.06	0.12	−0.05	0.31	−0.04	0.11	−0.01	0.21	−0.08	−0.04	−0.12	1.00
17	−0.08	−0.07	−0.17	−0.08	−0.08	−0.17	0.02	0.16	0.29	−0.09	−0.16	−0.06	−0.15	−0.23	−0.19	0.04	1.00
Total	0.53	0.47	0.36	0.28	0.48	0.29	0.51	0.16	0.16	0.56	0.55	0.38	0.57	0.18	0.49	0.12	−0.10	1.00

Note. Intercorrelations greater than 0.20 are in bold.

Table 3. Sample means and two-sample t-tests for Hamilton Depression Rating Scale items by Major Depressive Disorder status.

	Major Depressive Disorder		No Major Depressive Disorder
Item	Mean	SD	Mean	SD	Difference	t value	p value	CI
Item 1	1.84	0.91	0.76	1.04	1.08	4.48	<.01	[0.60, 1.56]
Item 2	1.74	0.79	1.05	0.97	0.69	2.94	.01	[0.27, 1.12]
Item 3	0.47	0.73	0.00	0.00	0.47	2.80	<.01	[0.15, 0.78]
Item 4	0.91	0.94	0.67	0.86	0.25	1.10	.28	[−0.22, 0.71]
Item 5	0.91	0.88	0.62	0.74	0.29	1.48	.15	[−0.14, 0.73]
Item 6	0.57	0.82	0.24	0.62	0.33	1.91	.06	[−0.06, 0.72]
Item 7	1.91	1.00	1.05	1.36	0.87	2.67	.01	[0.31, 1.43]
Item 8	0.05	0.22	0.10	0.30	0.04	0.6	.55	(−0.17, 0.08)
Item 9	0.91	0.76	1.10	1.04	0.18	0.73	.47	[−0.61, 0.24]
Item 10	2.14	0.89	1.24	1.00	0.90	3.65	<.01	[0.44, 1.36]
Item 11	1.40	1.09	0.52	0.98	0.87	3.39	<.01	[0.33, 1.41]
Item 12	0.60	0.79	0.29	0.56	0.32	1.98	.05	[0.06, 0.69]
Item 13	1.60	0.64	0.24	0.54	1.37	9.41	<.01	[1.05, 1.68]
Item 14	0.79	0.85	0.67	0.73	0.13	0.65	.52	[−0.29, 0.54]
Item 15	1.29	1.17	0.29	0.56	1.01	5.13	<.01	[0.48, 1.54]
Item 16	0.21	0.59	0.24	0.62	−0.03	0.20	.84	[−0.33, 0.27]
Item 17	0.10	0.36	0.38	0.80	−0.28	1.53	.14	[−0.54, 0.02]
Total score	17.48	4.26	9.43	3.92	8.05	7.88	<.01	[5.94, 10.17]

Note. Items in bold indicate nonsignificant finding at the 5% significance level.

SD = standard deviation; CI = confidence interval.

Internal consistency reliability

Internal consistency was evaluated using estimated intercorrelations of the items and Cronbach's alpha (Cronbach, 1951) calculated using data from all participants (N = 102). Separate analyses were not conducted by depression status, as sample sizes were too small to provide reliable estimates. The estimated intercorrelations in Table 2 indicate that the 17 items are weakly to moderately correlated with one another and are moderately correlated with the total score. The estimated value of Cronbach's alpha is 0.58, a subacceptable value (Tavakol & Dennick, 2011).

Contrasted group validity

Table 3 reports summary statistics for each item and total HAM-D score by depression status with those with unknown MDD status excluded. Based on two-sample t-tests, means for nine of the 17 items do not differ at the 5% significance level. Because we were more concerned about committing a Type II error, no multiple comparisons adjustment was employed when conducting these tests. These results indicate that the HAM-D is not able to differentiate between those with and without MDD for these items.

Factorial validity

Consistent with previous studies (cf. Bagby et al., 2004), factor analysis was conducted using principal component analysis with varimax rotation. Because the factor structure of the HAM-D differs across previous studies (cf. Bagby et al., 2004), potentially due to the heterogeneity of symptoms, the results of the factor analysis should be considered descriptive of this sample. Six factors with eigenvalues greater than 1 were extracted, which account for 60.92% of the total variability of the data. Table 4 reports the pattern matrix. The first factor is characterized as a general depression dimension with items (1) depressed mood, (2) feelings of guilt, (3) suicide, (7) work and activities, and (13) somatic symptoms–general having high factor loadings. Factor 2 represents an anxiety dimension, as items (10) anxiety–psychic, (11) anxiety–somatic, and (12) somatic symptoms–gastrointestinal have high factor loadings. The third factor consists of items (4) insomnia–early, (5) insomnia–middle, and (6) insomnia–late, representing a dimension of sleep difficulty. The remaining factors are difficult to interpret, which is attributed to extracting factors with eigenvalues greater than 1. Consistent with previous research, the HAM-D for this sample is multidimensional, with dimensions relating to general depressive symptoms, anxiety, and sleep difficulty.

Table 4. Rotated factor loadings.

	Factor 1	Factor 2	Factor 3	Factor 4	Factor 5	Factor 6
Item 1	0.71	0.19	−0.02	−0.02	−0.12	−0.06
Item 2	0.55	0.36	0.08	0.03	−0.17	−0.18
Item 3	0.50	−0.14	0.23	−0.52	0.03	−0.04
Item 4	−0.01	−0.08	0.68	−0.06	−0.14	0.07
Item 5	0.12	0.17	0.79	0.05	0.11	−0.10
Item 6	−0.08	0.25	0.47	−0.40	0.30	−0.06
Item 7	0.74	−0.14	−0.01	−0.13	0.27	0.15
Item 8	0.16	−0.22	0.00	0.27	0.78	0.16
Item 9	−0.02	0.03	0.06	0.81	0.05	0.00
Item 10	0.38	0.57	−0.02	0.15	0.16	0.15
Item 11	0.14	0.77	0.18	0.06	−0.17	0.04
Item 12	−0.03	0.73	−0.03	−0.13	0.13	0.02
Item 13	0.50	0.29	0.11	0.24	−0.03	0.41
Item 14	−0.07	0.01	−0.14	−0.02	0.12	0.80
Item 15	0.36	−0.01	0.37	0.14	−0.20	0.47
Item 16	−0.14	0.24	−0.02	−0.11	0.71	−0.13
Item 17	0.03	−0.21	−0.12	0.50	0.20	−0.53
Eigenvalue	2.97	1.72	1.59	1.50	1.34	1.24

Note. Bold values indicate items with factor loadings greater than 0.4 in absolute value.

Discussion

The authors report the results of what is, to the best of our knowledge, the first preliminary psychometric findings on the 17-item HAM-D in a population of methamphetamine users. The results of our study indicate potential concerns with construct validity of the HAM-D when administered to methamphetamine users. Internal consistency is questionable. Intercorrelations between items and with the total score greater than 0.20 and Cronbach's alpha values greater than 0.70 are considered necessary for adequate internal consistency (Briggs & Cheek, 1986; Nunnally & Bernstein, 1994; Tavakol & Dennick, 2011). From Table 2, only 31 of 136 intercorrelations of items are greater than 0.20, while 12 of the 17 items have a correlation of 0.20 or greater with the total score and the estimated Cronbach's alpha of 0.58. Taken together, these indicate issues with internal consistency. One potential reason for these results is that this sample consists of methamphetamine users, and how the items relate to one another may not be the same as in a population of those with only MDD or due to heterogeneous constructs (Tavakol & Dennick, 2011). Contrasted group validity is also questionable. Although average overall scores differ for those with and without MDD, the means for 9 of the 17 items do not differ statistically at the 5% significance level. These results suggest concerns with specific items of the HAM-D. This likely owes to the similarity in depressive symptoms and side effects of methamphetamine. For example, the means for the items associated with insomnia do not differ, which may be due to insomnia being a common symptom of depression and methamphetamine use. This indicates that certain items of the HAM-D should be reevaluated when using this instrument in a population of methamphetamine users.

The results of the factor analysis indicate potential issues with factorial validity. Consistent with previous studies (Bagby et al., 2004), for this sample, the HAM-D is multidimensional. The factor structure of the HAM-D for this sample has several dimensions similar to previous studies including dimensions of general depression, anxiety, and sleep difficulty. What differs is that a factor that is a combination of depression and anxiety was not identified in this sample. Further, the six extracted factors explain 60.92% of the variability in the data, indicating that this factor structure may not adequately describe the data. Due to differences in how depression is manifested in methamphetamine users, the factor structure may differ for this population compared to those considered in previous studies. These results indicate that additional research is warranted to identify the factor structure of the HAM-D in methamphetamine users.

Because of the preliminary nature and small sample size of our study, we are not suggesting a change in practice for clinicians or researchers using the HAM-D based on our findings. There is limited knowledge regarding the management of comorbid depression and methamphetamine use disorders (Hellem et al., 2015), and our findings suggest the need for a broader evidence-based foundation for screening and assessing patients with comorbid depression and methamphetamine use disorders. Implications for future research include more testing of the HAM-D in larger samples of methamphetamine users to better understand psychometric properties. We believe that the HAM-D is an excellent clinical tool for detecting changes in depression severity, and until we learn more about how the HAM-D functions in methamphetamine users, it is an acceptable depression severity tool to use.

Limitations

There are several limitations to this study. First, although data were used from four separate studies, the sample was homogeneous with respect to gender and race. More women were included in our analysis than men because two of the studies (Hellem et al., 2015; ClinicalTrials.gov ID NCT02192931) from which we extracted data recruited women only. Second, the retrospective nature limits generalizability to other patient populations. Third, methamphetamine abstinence data and methamphetamine urine drug screening results were not collected from participants who screened out of the four studies. These data would be useful for understanding intoxication versus withdrawal at the time of HAM-D administration. In addition, route of administration of methamphetamine was not collected. Because research demonstrates that psychiatric symptoms may fluctuate depending on route of administration (Rawson, 2013), this may have influenced the severity and course of symptoms of depression.

Another limitation is reliably diagnosing active substance use with MDD. Although trained personnel administered the SCID-IV and PRISM, evidence suggests that assessing psychiatric illness in the presence of an ongoing substance use disorder can be complicated by symptoms of intoxication and withdrawal (Stein et al., 2004; Langås et al., 2011; Wynn, Landheim, & Hoxmark, 2013). However, previous research has shown that PRISM, which was used to assess DSM-IV disorders in two of the studies, increases the reliability of diagnosing psychiatric illnesses in substance users (Hasin, Trautman, & Endicott, 1998; Hasin et al., 2006).

Conclusions

In this small sample of adults who use methamphetamine, our early data suggest that the 17-item HAM-D may not adequately measure severity of depression considering the potential concerns we found with construct validity. These findings may represent symptoms of methamphetamine withdrawal or the lack of sensitivity of the 17-item HAM-D in our sample. Further prospective research is warranted in larger samples of individuals with and without an MDD diagnosis who use methamphetamine.

Disclosures

Tracy Hellem contributed to research conception and design and interpretation of results and took the lead on preparation and draft revision. Lindsay Scholl contributed to University of Utah IRB correspondence and data extraction. Hayden Ferguson contributed to data extraction. Erin McGlade and Deborah Yurgelun-Todd provided data from one of the studies from which we extracted data and contributed to editing the revised manuscript. Perry Renshaw contributed to editing. Laura Hildreth contributed to research design, statistical programming and analysis, interpretation of results, and writing and draft revision.

The National Institutes of Health (NIH) and the Veteran's Affairs Medical Research Program have supported Drs. Yurgelun-Todd's and Renshaw's research. Dr. Renshaw has been a consultant to Kyowa Hakko, Tal Medical, and Ridge Diagnostics. The views and opinions contained within this manuscript do not necessarily reflect those of the NIH or Veteran's Affairs Medical Research Program. All other authors declare that they have no conflicts of interest.

The following lists compensation for professional services in any of the previous three years for each author:

Tracy Hellem: (1) University of Utah; (2) Westminster College; (3) Montana State University; and (4) Chamberlain College of Nursing.

Lindsay Scholl: (1) University of Utah and (2) Northeastern University.

Hayden Ferguson: (1) Montana State University.

Erin McGlade: (1) University of Utah; (2) Veteran's Affairs Medical Center in Salt Lake City.

Deborah Yurgelun-Todd: (1) University of Utah; (2) Veteran's Affairs Medical Center in Salt Lake City; (3) National Institute of Health; (4) Utah Science Technology and Research Initiative; and (5) Kyowa Hakko.

Perry Renshaw: (1) University of Utah; (2) Veteran's Affairs Medical Center in Salt Lake City; (3) National Institute of Health; and (4) Utah Science Technology and Research Initiative.

Laura Hildreth: (1) Montana State University.

Acknowledgment

This work was presented at the fifth annual meeting of Addiction Research & Therapy, Atlanta, GA, in October 2016.

Funding

Tracy Hellem received funding from The College of Nursing and Office of Research and Economic Development at Montana State University to support this project.