Psychometric validity of the Montgomery and Åsberg Depression Rating Scale for Youths (MADRS-Y)

Abstract

Background

Because of all the serious consequences of major depressive disorder (MDD), it is important to screen for MDD in adolescents. The aim of this study was to test the psychometric properties of the newly developed self-report depression scale MADRS-Y for adolescents in a normative Swedish sample.

Methods

The study included 620 adolescents in the age range of 12–20 years old. The normative sample was randomly split into two equal parts, to perform principal component analysis (PCA) on sample one and confirmatory factor analysis (CFA) on sample two. We investigated the psychometrics.

Results

The result from the PCA suggested that all 12 potential items should be used, and the items loaded on the same construct of depression. The CFA supported the one-factor structure with good fit indices. Measurement invariance was confirmed, allowing interpretation regardless of gender or age differences. Reliability was good, α .89, for both samples separately. Test-retest reliability was good to excellent (intraclass correlation coefficients = .87 and .91). Evidence of convergent and discriminant validity was shown.

Conclusions

The results in the current study suggest that the MADRS-Y is a brief, reliable, and valid self-report questionnaire of depressive symptoms for adolescents in the general population.

Keywords:

• Depression
• adolescents
• confirmatory factor analysis
• scale development
• psychometrics

Major depressive disorder (MDD [1]) in adolescence is a serious and common global health problem [2,3]. The probability of depression rises from around 5% in early adolescence to as high as 20% at the end of adolescence [3]. For untreated depression in adolescence, there is an increased risk of suicide [3], substance use disorders [4], and numerous negative psychosocial consequences and impairments [5–7]. Because of all the serious consequences of MDD, it is important to screen for and to evaluate the treatment effect of depressive symptoms. In the current study, we investigated the psychometric properties of the self-report questionnaire Montgomery – Åsberg Depression Rating Scale – Self report (MADRS-S) [8] adapted for adolescents; Montgomery – Åsberg Depression Rating Scale –Youth (MADRS-Y).

Self-report questionnaires for depression are time efficient and require few staff resources [9]. They can be especially useful when multiple assessments are needed [10]. Adolescent self-assessments result in more internalizing symptoms compared to parent ratings [11], and there is agreement that adolescents could report their internal state more accurately than other informants [12]. Patients’ self-ratings can, in some areas, outmatch clinicians’ ratings, for example in the assessment of suicidality [13].

Many of the commonly used MDD self-report questionnaires for children and adolescents have not been sufficiently psychometrically evaluated for use in this age group [14]. Questionnaires for adolescents are often inadequate, with a childhood or adult perspective, without considering the unique development period in adolescents [15]. In addition, it is important that instructions and item descriptions are age-adjusted and easy to understand for adolescents [16].

The criteria for MDD in the Diagnostic and Statistical Manual of Mental Disorders (5th ed.; DSM–5) for adolescents and adults are identical, with the important exception that irritability counts as a core symptom for adolescent MDD and can replace depressed mood [1]. In a study with male adolescents (n = 1004, aged 18 years), the symptom ‘irritability’ appeared to be the best indicator of male depression [17], and it has also been shown to be one of the most common symptoms in girls [18]. Adolescents diagnosed with MDD report both symptoms of insomnia and hypersomnia [19], which are both included in the sleep disturbance DSM-5 criteria for MDD, and they report weight changes and appetite disturbances in both directions [20–22], which is also included in DSM-5 MDD criteria.

To better capture adolescent MDD [17,19–23] and the DSM-5 criteria [1], we developed an adapted version of the adult MADRS-S. The content of the items in MADRS-Y are compatible with the DSM-5 diagnostic criteria for MDD, except the item worry/anxiety, which is not part of the DSM-5 MDD criteria, even though the DSM-5 describes anxiety as a prominent feature of MDD [1]. Another difference is that the DSM-5 has a MDD criteria describing psychomotor disturbance, which is not included in the MADRS-Y, since it is much less common [1].

The aim of this study was to test the psychometric properties of the newly developed self-report scale in Swedish, MADRS-Y, for adolescents in a normative sample.

Method

Study setting

Data was collected in the project called Adolescents’ experience of mental illness – psychometric properties of new Swedish versions of tests (UPOP). The purpose of the project was to develop new instruments for screening and to evaluate treatment effects of mental illness among young patients. The project was approved by the Swedish Regional Ethical Review Board in Umeå (number 2018/59-31). The study was conducted in a medium-sized town (90 000 inhabitants) and its surroundings in the northern part of Sweden. Data collection took place during the years 2018 to 2019.

Procedure

The MADRS-Y questionnaire was developed from the original version MADRS-S by our research group. Instructions and item descriptions were age-adapted throughout the questionnaire (shorter sentences, easier and modernized language) to be easily understood and administered. We also adjusted the MADRS-Y questionnaire to the DSM-5 MDD criteria [1]. Four adolescents (15–18 years old) filled in the MADRS-Y questionnaire and gave feedback through interviews before we distributed the form to the participants in this study. No changes were made based on the interviews. Four schools from different socioeconomic areas were included. The students were asked to fill out the self-report questionnaire online. Verbal and written information was given to students by research assistants, and written consent was then obtained from the students who chose to participate. For the students younger than 15 years of age, informed consent was also obtained from the parents. During the time of participation, a research assistant and a teacher were present to answer questions from the participants. The procedure has been described in detail elsewhere [24].

Participants

The study population included a convenience sample with students from four schools in the age-range of 12–20 years old. The schools for the older age group offered different programs. Exclusion criteria were 1. lack of fluency in written Swedish and 2. inability to complete online forms. Of the 897 students who were asked to participate in the study, 620 (69%) choose to do so. 620 respondents were included in the analyses, of which 383 (61.8%) were girls. Most of the respondents were born in Sweden (88%).

In order to avoid performing principal component analysis (PCA) and confirmatory factor analysis (CFA) on the same sample, the sample was randomly split into two equal parts [25]: sample one (n = 310) and sample two (n = 310). For measuring socioeconomic status, a Swedish socioeconomic classification system [26,27] was used to estimate the households’ places in a socioeconomic ranking based on six different classes of parents’ socioeconomic status, see Table 1.

Table 1. Descriptive statistics for MADRS-Y.

	Sample one N = 310				Sample two N = 310				Significance test		Effect size Cohen’s d
Items	Mean	SD	Frequency or range	%	Mean	SD	Frequency or range	%	p	X^2	d
Sex – girls			187	60.3			196	63.2		.457
Age	15.79	1.75	12–20		15.68	1.80	12–20		.443		.06
Parents’ socioeconomic status
Students							2	.6
Manual workers			57	18.4			53	17.1
Clerical or office workers			86	27.7			95	30.6
Higher civil servants			103	33.2			104	33.5
Self-employed			21	6.8			24	7.7
Never used alcohol			201	64.8			206	66.5
Never used drugs			295	95.2			299	96.5
Difficulties (emotional, concentration, behaviour, or relational) (0 = no; 3 = yes, serious difficulties)	0.63	0.76			0.65	0.79			.756		–.02
MADRS-Y total scale	11.39	10.18					12.33	10.66	.262		–.09
MADRS-S total scale (for adults)	9.78	8.16					10.05	8.64	.692		–.03
RADS-2 total scale	1.98	0.50					1.99	0.54	.886		–.01
RCADS Anxiety total scale	21.85	14.60					23.60	17.00	.169		–.11
WHO-5 total scale	59.10	21.73					59.00	21.64	.955		.00
PROMIS Pediatric Short Form v.1.0—Family relationships 8a	25.60	7.54					25.96	6.62	.526		–.05
PROMIS Pediatric Bank v2.0 – Peer relationships	44.61	11.20					44.93	9.89	.707		–.03
PROMIS Pediatric Bank v1.0—Physical activity	19.58	8.98					20.22	8.92	.375		–.23

Note. M = mean, SD = standard deviation, X² = Chi Square test, p = significance of t-test.

The adolescents were asked to complete the same self-reports again after a period of three weeks to obtain data on test-retest reliability. 173 (55.8%) adolescents from sample one and 175 (56.5%) from sample two completed the re-test. In sample one, there were 117 (67.6%) girls, and their ages ranged from 12 to 18 years old (M = 15.46 SD = 1.70). In sample two, there were 121 (69.1%) girls, and their ages ranged from 12 to 19 years old (M = 15.34 SD 1.65).

Measures

All measures in the current study are self-reporting questionnaires in Swedish.

Description of the self-assessment measure MADRS-Y

Montgomery – åsberg depression rating scale – youth (MADRS-Y)

MADRS-Y is a brief self-report questionnaire for adolescents including 12 items. It has been developed from the MADRS-S [8] and measures symptoms of depression. As in MADRS-S, the items assess reported sadness, anxiety, reduced sleep, reduced appetite, concentration difficulties, lassitude, inability to feel, pessimistic thoughts, and suicidal thoughts. However, in MADRS-Y, there are also three new items: irritability, increased sleep, and increased appetite. The self-report is measured with a scale from 0 (low) to 6 (high), and the maximum score is 72. Higher scores indicate more severe depression. MADRS-Y begins with instructions including how the rater is feeling right now and during the last three days.

Convergent validity

Montgomery – Åsberg Depression Rating Scale – Self report (MADRS-S)

MADRS-S is an adult self-report questionnaire of depression symptom [8,28]. The scale includes nine items on a scale from 0 (low) to 6 (high). The maximum score is 54, and higher score indicates more severe depression [28]. MADRS-S is a reliable and valid instrument for adults with good sensitivity for assessing MDD. The internal consistency measured with Cronbach’s alpha has shown to be between .76 and .94 [29]. Only one study has validated MADRS-S in an adolescent psychiatric sample [29], it was found that the psychometric properties and diagnostic accuracy of the MADRS-S (n = 105, aged 12–17 years) were good (e.g. Cronbach alpha .87). In the current study, Cronbach’s alpha was .84 in sample one and .87 (95% CI) in sample two.

WHO-five well-being index (WHO-5)

WHO-5 [30] is a short global rating scale with five statements measuring subjective well-being [31,32]. The respondent is asked to rate how well 5 statements applies considering the last 14 days. Each item is scored from 0 (not present) to 5 (constantly present), where higher scores mean better wellbeing [32]. The scale has shown adequate validity [32]. The WHO-5 has demonstrated acceptable sensitivity and good specificity compared to other depression screeners for adolescents, as well as good psychometric qualities (Cronbach’s alpha .85) [33]. Cronbach’s alpha in the current study was .86 for sample one and .88 for sample two (95% CI).

The revised child anxiety and depression scale (RCADS)

The RCADS [34] is a scale for children and adolescents measuring anxiety and depression. RCADS contains 47 items, of which 37 items measure symptoms of anxiety and 10 items measure symptoms of depression. Items are scored 0 (never) to 3 (always) [34]. Some items were not collected due to a mistake when programming the web survey, therefore, we used only the anxiety scale. The mean alpha value for the anxiety scale has been found to be the same as for the total score (.93) [35]. It has been shown to be a valid and reliable instrument (e.g. Cronbach’s alpha of .96) [36]. In the current study, Cronbach’s alpha was .89 for sample one and .97 for sample two (95% CI).

Reynolds adolescent depression scale second edition (RADS-2)

RADS-2 [37] is a self-report questionnaire measuring depressive symptoms in adolescents. The scale includes 30 items and each item scores from 0 (almost never) to 4 (most of the time). Higher scores indicate higher symptom severity. The RADS-2 is a well validated self-report questionnaire and has also been shown to be a reliable instrument to use in a Swedish community setting [38]. Cronbach’s alpha in the current study was .87 for sample one and .88 for sample two (95% CI).

Divergent validity

Patient-reported outcomes measurement information system (PROMIS)

PROMIS, developed by The US National Institutes of Health, is designed to develop, validate, and standardize item banks to measure patient-reported health [39]. The following PROMIS item banks [39,40] were used to investigate divergent validity: PROMIS Pediatric Item Bank v2.0 - Peer Relationships (15 items) [41], PROMIS® Pediatric Item Bank v1.0 - Family Relationships - Short Form 8a (8 items) [42,43], and PROMIS® Pediatric Item Bank v1.0 - Physical Activity (10 items) [44–46]. These three item banks were chosen because they are all related, but different, to the construct depression. All PROMIS forms use a five-point Likert scale ranging from 1 (e.g. ‘never’ or ‘not at all’) to 5 (e.g. ‘always’ or ‘almost always’). The following are Cronbach’s alphas in the current study for sample one: peer relationships .93, family relationships .95, and physical activity .94, and for sample two: peer relationships .91, family relationships .95, and physical activity .94 (95% CI).

Statistical analysis

We randomly split the sample into two equal halves (two samples with n = 310), using SPSS, version 27.0. Thereafter we analyzed the samples separately in all analyzes. In sample one, a PCA was performed to explore the factor structure. In sample two, a CFA was calculated to examine whether the same factor structure was obtained as in the PCA when using a new sample.

Descriptive statistics were calculated first. T-tests were used [47] for gender, age, and socioeconomic differences, and Cohen’s d for effect sizes. Cohen’s guidelines for effect size are: d ≥ .2 a small effect, d ≥ .5 a medium effect, and d ≥ .8 a large effect [48]. The dividing into the two age groups 12–15 years and 16–20 years, is the same dividing as in the Swedish school system.

We calculated corrected item-total correlations ($r_{\text{it}}^{\mathrm{c}}$) and a correlation less than .3 indicates that the item does not correlate well with the overall scale and should be removed [49,50]. Inter-correlations between items calculated by Pearson’s r. Cohen’s guidelines for effect size are: r ≥ .1 a small effect, r ≥ .3 a medium effect, and r ≥ .5 a large effect [48].

The reliability of the scale was calculated by using Cronbach’s α, and an α between .70 and .90 is considered to be good [51].

Test–retest reliability over a 3-week period of the MADRS-Y was estimated using intraclass correlation coefficients (ICCs). ICC estimates and their 95% confident intervals were calculated with absolute agreement type and a 2-way mixed model. Values less than .5 are indicative of poor reliability, values between .5 and .75 indicate moderate reliability, values between .75 and .9 indicate good reliability, and values greater than .9 indicate excellent reliability [52].

To select the final items for the MADRS-Y, we analyzed all 12 potential items using PCA on sample one. PCA is used to explore which factors or items are loading on the same constructs and it is used to obtain more simple or interpretable factors [53]. The significance is dependent on the sample size. With a sample of more than 300 participants, loadings of .30 should be significant [54]. We used an oblique rotation (Oblimin in SPSS) in the PCA, because we had intercorrelations between almost all items. First, the assumptions for the PCA model were tested. Kaiser-Meyer-Olkin (KMO) was used to test for sampling adequacy. The KMO varies between 0 and 1, and values smaller than 0.5 will indicate a too small sample [55]. Bartlett’s test should be significant to show that correlations between variables are significantly different from zero. However, this test is dependent on sample size and is often significant. Correlations between items should not be too high. If they are above .80 [50], it is an indication that they are measuring the same thing.

A Scree plot and eigen values were calculated. Criteria are that each principal component explain at least 5% of the variance, the cumulative variance be at least 7%, and eigenvalue criteria must be more than 1 [56].

To confirm the latent structure of the MADRS-Y in sample two, we used a single factor CFA. The internal consistency was examined using Lavaan package for structural equation modelling version 0.6 − 3 (BETA [57];. Fit indices used were: the Satorra–Bentler Chi square (SB χ2) and the coefficient χ2/df (≤ 3.00 [58]), the comparative fit index (CFI; > .90 [59];), the Tucker–Lewis index (TLI; > .90 [59];), the root mean square error of approximation (RMSEA < .08 [60]), and standardized root mean square residual, SRMR < .08 [61]. A robust diagonally weighted least squares with scaled-shifted test (DWLSSS) estimator was used because we had a non-normal distribution and ordinal scale responses [62]. Diagonally weighted least squares (DWLS) use a polychoric correlation matrix, not sensitive for non-normal distribution.

Measurement equivalence or invariance across sex and age groups was calculated. First, a baseline model or configural invariance model was estimated [63]. Second, a test of invariant factor loadings or metric invariance was done. Third, invariant factor covariance or scalar invariance was calculated. When examining Goodness of fit, it is recommended to not only use Chi square (X²), because it is sensitive to sample sizes [64]. Thus, we also used ΔRMSEA, ΔCFI, and Satorra-Bentler’s test. Our interpretations followed the recommendations for small and skewed data: ΔRMSEA ≤ .01 can be interpreted as an excellent fit (close fit: .01–.05; fair fit: .05–.08; mediocre fit: .08–.10; poor fit: .10+) [65]. For a summary of different goodness of fit indices refer to [63]. A CFI change of smaller than or equal to −.01 is evidence of measurement invariance (null hypothesis of invariance should not be rejected) even with 150 per group [65,66]. A non-significant Satorra-Bentler test indicates that there is no difference between the models [63].

We calculated evidence of the convergent and divergent validity by using Pearson’s correlations (r) between MADRS-Y and the convergent and divergent validity measures.

A variable correlation plot was performed and principal component analyses were used for showing the relationships between the MADRS-Y and all convergent and divergent validity scales. Positively correlated variables were grouped together, and negatively correlated variables were positioned on opposite sides in the plot [67].

Results

Descriptive statistics of sample one and two

Descriptive statistics were calculated for sample one and two separately, see Table 1. Both samples were equally sized (n = 310) and did not differ significantly in gender, age, SES, symptoms on MADRS-Y, or for all validity scales.

Gender, age, and socioeconomic differences

T-tests showed that girls rated significantly more depressive symptoms than boys on the MADRS-Y total scale in sample one (p = .032) and in sample two (p < .001), with small effect sizes. See Table 2.

Table 2. Internal consistency and T-test between gender, for the Swedish sample of MADRAS-Y total score.

	Boys			Girls			Total			T-test
MADRS-Y total score	M	SD	α	M	SD	α	M	SD	α	T	p	Cohen’s d
	n = 123			n = 187				n = 310
Sample One	8.64	8.26	.88	12.40	10.33	.89	11.39	10.18	.89	2.16	0.032*	0.25
	n = 114			n = 196				n = 310
Sample Two	9.46	10.04	.89	14.00	10.67	.89	12.33	10.66	.89	3.69	0.000***	0.44

Note. M mean, SD standard deviation, α = Cronbach alpha. *= p < .05, *** < p = .001. Effect size Cohen’s d ≥ .2 small effect, ≥ .5 medium effect, and ≥ .8 large effect.

For age differences the samples were divided into the two groups 12–15 years old (sample one N = 134, sample two N = 148) and 16–20 years old (sample one N = 176, sample two N = 162). In sample one 12–15 years old rated significantly (p < .001) less depressive symptoms (M = 8.81 SD = 9.54) than 16–20 years old (M = 13.35, SD = 10.24) with small effect size (d = –.46, 95% CI (–.68 to –.23)). In sample two there was no significant difference (p = .235) between the age groups 12–15 years (M = 11.57, SD = 11.99) and 16–20 years old (M = 13.02, SD = 9.26) with small effect size (d = –.137, 95% CI (–.36 to .09)).

For socioeconomic differences the samples were divided into two groups (group 1: students, manual workers and clerical or office workers (sample one N = 143, sample two N = 150) and group two: higher civil servants, and executives and self-employed in the other (sample one N = 124, sample two N = 128). There was no significant difference (p = .094) in scores between group one (M = 10.23, SD = 8.98) and group two (M = 12.21, SD = 10.28) in sample one with small effect size (d = –.21, 95% CI (–.45 to .04)). Also in sample two, there was no significant difference (p = .531) in scores between group one (M = 12.61, SD = 10.53) and group two (M = 11.82, SD = 10.28) with small effect size (d = –.08, 95% CI (–.16–.31)).

Descriptive statistics and corrected item-total correlation of the items

Before calculating PCA and CFA, descriptive statistics of the items were obtained, see Table 3. The corrected item-total correlation (${\mathrm{r}}_{\text{it}}^{\mathrm{c}}$) was higher than .3 in both samples. The items that measured increased sleep and increased appetite had lowest ratings in mean and in item total correlation (.32– .43) in both samples, however, they contributed to ordinal alpha.

Table 3. Descriptive statistics for MADRS-Y.

	Sample one N = 310				Sample two N = 310
Items of MADRS-Y	Mean	SD	${\mathrm{r}}_{\text{it}}^{\mathrm{c}}$	α-i	Mean	SD	${\mathrm{r}}_{\text{it}}^{\mathrm{c}}$	α-i
1. Sadness	0.91	1.34	0.74	0.87	0.89	1.37	0.64	0.88
2. Irritability	0.98	1.32	0.60	0.88	1.18	1.48	0.66	0.88
3. Anxiety	1.59	1.61	0.69	0.87	1.61	1.60	0.67	0.88
4. Decreased sleep	1.27	1.47	0.56	0.88	1.24	1.37	0.50	0.89
5. Increased sleep	0.57	1.02	0.36	0.89	0.74	1.25	0.43	0.89
6. Decreased appetite	0.61	1.04	0.54	0.88	0.65	1.02	0.49	0.89
7. Increased appetite	0.51	1.15	0.38	0.89	0.48	0.99	0.32	0.90
8. Concentration	1.21	1.45	0.64	0.87	1.39	1.51	0.72	0.88
9. Starting to do things	1.55	1.39	0.60	0.88	1.63	1.35	0.66	0.88
10. Interest and engagement	0.79	1.08	0.62	0.88	0.83	1.06	0.68	0.88
11. Thoughts about yourself and your future	0.81	1.16	0.71	0.87	0.95	1.33	0.76	0.88
12. Life or death thoughts	0.59	1.03	0.68	0.87	0.73	1.23	0.71	0.88

Note. M = mean, SD = standard deviation, ${\mathrm{r}}_{\text{it}}^{\mathrm{c}}$ = corrected item-total correlation, α–i = ordinal alpha if the item is removed.

Correlations between items in sample one

Pearson’s correlations showed that most correlations were larger than .32, indicating the correct use of an oblique (correlating) rotation in the PCA. 47 correlations ranged from .32 to .74, and 19 correlations ranged from .14 to .31. All correlations were significant at the 0.01 level. Inter-correlations between MADRS-Ýs items are seen in Table 4.

Table 4. Intercorrelations between MADRS-Y items using Pearson’s r for sample one and sample two.

Measure	1.	2.	3.	4.	5.	6.	7.	8.	9.	10.	11.	12.
1. Sadness	1.000	.478**	.657**	.436**	.251**	.473**	.312**	.479**	.419**	.567**	.673**	.602**
2. Irritability	.535**	1.000	.504**	.350**	.266**	.409**	.321**	.456**	.388**	.419**	.406**	.450**
3. Anxiety	.575**	.526**	1.000	.409**	.241**	.402**	.243**	.506**	.544**	.455**	.574**	.487**
4. Decreased sleep	.316**	.358**	.369**	1.000	.414**	.445**	.205**	.395**	.340**	.415**	.383**	.405**
5. Increased sleep	.210**	.259**	.322**	.367**	1.000	.267**	.226**	.263**	.208**	.243**	.184**	.157**
6. Decreased appetite	.338**	.327**	.392**	.286**	.248**	1.000	.141*	.356**	.285**	.336**	.423**	.462**
7. Increased appetite	.171**	.225**	.166**	.083	.345**	.148**	1.000	.251**	.290**	.236**	.365**	.303**
8. Concentration	.419**	.551**	.477**	.451**	.340**	.340**	.373**	1.000	.633**	.405**	.465**	.438**
9. Starting to do things	.444**	.540**	.486**	.413**	.211**	.330**	.231**	.681**	1.000	.391**	.440**	.381**
10. Interest and engagement	.507**	.491**	.484**	.352**	.341**	.367**	.246**	.569**	.512**	1.000	.522**	.569**
11. Thoughts about yourself and your future	.591**	.532**	.603**	.393**	.303**	.454**	.229**	.596**	.554**	.568**	1.000	.736**
12. Life or death thoughts	.599**	.507**	.505**	.339**	.371**	.463**	.271**	.513**	.452**	.553**	.708**	1.000

Note. *p < 0.05, **p < 0.01. Sample One is in the right upper corner and Sample Two is in the left under corner.

Principal component analysis in sample one

One PCA analysis with oblique rotation (direct oblimin) was conducted with the 12 items on sample one. The Kaiser-Meyer-Olkin measure verified the sampling adequacy for the analysis, KMO = .88, which is well above the acceptable limit of 0.5 [55]. Bartlett’s test was significant. Initial analysis was run to obtain eigenvalues for each factor in the data. Two factors had eigenvalues over Kaiser’s criterion of 1, and the first explained 44.86% variance while the second explained 9.11% variance in the PCA. The scree plot justified to retain only one factor, a depression factor. Table 5 shows the factor loadings of the PCA.

Table 5. Principal Component Analysis (PCA) (Sample One) and Confirmatory Factor Analysis (CFA) (Sample Two) Results for the MADRS-Y.

	PCA in Sample One (N = 310)		CFA in Sample Two (N = 310)
	Rotation method: Oblimin		DWLSSS Robust
Item number and meaning of item	Factor 1	Factor 2	First Model’s factor loadings	Second Model’s factor loadings
1. Sadness	0.64	–0.26	0.78	0.79
2. Irritability	0.71	0.12	0.74	0.75
3. Anxiety	0.67	–0.36	0.75	0.76
4. Decreased sleep	0.71	–0.09	0.59	0.60
5. Increased sleep	0.54	0.51	0.50	0.51
6. Decreased appetite	0.51	–0.02	0.58	0.58
7. Increased appetite	0.42	0.74	0.41	0.41
8. Concentration	0.72	–0.11	0.83	0.77
9. Start to do things	0.69	–0.23	0.78	0.71
10. Interest and engagement	0.81	0.09	0.75	0.76
11. Thoughts about yourself and your future	0.74	–0.06	0.85	0.87
12. Life or death thoughts	0.76	0.01	0.84	0.85

Note. Showing the PCA and CFA model’s factor loadings. Factor loadings over 0.40 appear in bold.

Correlations between items in sample two

Pearson’s correlations showed that correlations ranged from .15 to .71. There was only one pair that did not correlate (increased appetite and decreased sleep, r = .08). Inter-correlations between MADRS-Ýs items are seen in Table 4.

Confirmatory factor analysis in sample two

A CFA of the MADRS-Y yielded a one-factor model for the Swedish normative sample (sample two). The first analysis showed varying results of the model to the data, χ2/df and RMSEA was higher than recommended goodness of fit indices: χ2 (54) = 205.68, χ2/df = 3.81, CFI = .96, TLI = .95, RMSEA = 0.10 (90% CI .08 to .11; SRMR = .06). The scale showed standardized factor loadings higher than .40 for all items, see Table 5.

Measurement invariance in sample two

In testing measurement of invariance for gender, the configural invariance model demonstrated an acceptable fit to the data indicating that the factor structure of the MADRS-Y was equivalent for boys and girls. The metric invariance model showed a good fit as seen in Table 6, except for ΔRMSEA, which had a close fit, indicating that items are loading in the same way for boys and girls, despite the constraint of ‘thresholds’. The scalar invariance model showed a good fit (‘thresholds’ and ‘loadings’ where constrained) indicating that there are no observable sex differences between the intercepts in the models. Satorra-Bentler’s tests were not significant, which indicating a good model.

Table 6. Measurement Invariance Goodness of Fit for the One-Factor Model of MADRS-Y.

Invariance	$\chi$^2(df)	CFI	TLI	RSMEA (90% CI)	$\Delta \chi$	$\Delta df$	$\Delta$ CFI	$\Delta$ RSMEA
Sex
Configural	226.965** (106)	0.972	0.965	0.086 (0.071–0.102)
Metric	266.285** (152)	0.974	0.977	0.070 (0.056–0.084)	–39.32	–46	–0.002	0.016 close fit
Scalar	254.720** (163)	0.979	0.983	0.060 (0.046–0.074)	11.57	–11	–0.003	0.01
Age group
Configural	259.728** (106)	0.969	0.962	0.097 (0.082–0.097) close fit
Metric	303.956** (152)	0.969	0.973	0.081 (0.067–0.094)	–44,228	–46	–0.006	0.016 close fit
Scalar	289.404** (163)	0.975	0.979	0.071 (0.057–0.084)	14,552	–11	–0.006	0.01

Note. ** = p < .001. χ2 = Chi square; df = degrees of freedom; CFI = Comparative Fit Index; RMSEA = Root Mean Square Error of Approximation; CI = Confidence Interval; TLI = Tucker-Lewis Index; ΔCFI = Change in Comparative Fix Index. For sex groups (girls and boys) Satorra-Bentler $\Delta \chi$²: Config vs. Metric $\chi$² (46) = 16.63, p = 1.00. Metric vs. Scaler $\chi$² (11) = 5.61, p = 0.90. For age groups: Satorra-Bentler $\Delta \chi$²: Config vs. Metric $\chi$² (46) = 20.22, p = 0.99. Metric vs. Scaler $\chi$² (11) = 6.13, p = 0.86.

Measurement invariance for age

When testing measurement invariance for age, two age groups were used in the calculations: 12 to 15 years old (n = 151, M = 14.13, SD = .19), and 16 to 20 years old (n = 108, M = 17.16, SD = 1.04).

The configural invariance model demonstrated a close fit to the data as seen in Table 6, only RMSEA was too high (.097), indicating that the factor structure of the MADRS-Y was possibly equivalent for the two age groups. The metric invariance model and the scalar invariance model showed a good fit, except for ΔRMSEA, which had a close fit, indicating that the factor loadings of the items and the intercepts were the same between the two age groups. Satorra-Bentler’s tests were not significant, which indicating a good model.

Internal consistency and test-retest reliability

Internal consistency for MADRS-Y was good (α =.89 in both samples separately) as seen in Table 2. In addition, internal consistency by gender was also good.

For test-retest reliability, ICC estimates were calculated. For sample one (n = 173), the three-week test-retest ICC was good .87 (95% CI .82, .90; p $<.$001), and for sample two (n = 175), the ICC was excellent .91 (95% CI .87, .93; p $<.$001).

Convergent and divergent validity

Table 7 displays the convergent and divergent validity. The result for convergent validity showed high correlations between all measures and MADRS-Y in both sample one (ranging from −.61 to −.90) and sample two (ranging from −.73 to −.90). For divergent validity the results for sample one (ranging from −.32 to −.41) and sample two (ranging from −.21 to −.43) showed small to moderate significant associations between the MADRS-Y and the three scales.

Table 7. Pearson’s correlations between MADRS-Y and validity measures.

Measure	MADRS-S	RADS-2	RCADS Anx	WHO-5	PROMIS Family	PROMIS Friends	PROMIS Physical Activity
MADRS-Y Sample One	.895*	.715*	.608*	–.730*	–.406*	–.392*	–.325*
MADRS-Y Sample Two	.899*	.753*	.736*	–.730*	–.417*	–.430*	–.212*

Note. *p < .01.

MADRS-S: Montgomery and Asberg Depression Rating Scale – Self-rated [8]; RADS-2: Reynolds Adolescent Depression Scale Version 2 [37]; RCADS Anx: The Revised Child Anxiety and Depression Scale – total Anxiety scale [34]; WHO-5: WHO Well-being Index [31]; PROMIS Family = PROMIS Pediatric Short Form v.1.0—Family relationships 8a; PROMIS Friends = PROMIS Pediatric Bank v2.0 – Peer relationships; PROMIS Physical Activity = PROMIS Pediatric Bank v1.0—Physical activity [45].

A PCA was conducted to visualize the correlations between the MADRS-Y and the validity scales, see Figure 1. The analysis showed that MADRS-Y had a strong positive correlation to the depression scales MADRS-S and RADS-2, and the anxiety scale RCADS anxiety. The MADRS-Y had a strong negative correlation to wellbeing (WHO-5). Furthermore, the analysis demonstrates clearly that the MADRS-Y measures a different construct than the PROMIS scales.

Figure 1. Principal Component Analyses of MADRS-Y total scale and all validity measures. Note. MADRS-S = Montgomery and Asberg Depression Rating Scale – Self-rated [8]; RADS-2 = Reynolds Adolescent Depression Scale Version 2 [32]; RCADS Anx = The Revised Child Anxiety and Depression Scale - total Anxiety scale [34]; WHO-5 = WHO Well-being Index [31]; PROMIS Family = PROMIS Pediatric Short Form v.1.0—Family relationships 8a; PROMIS Friends = PROMIS Pediatric Bank v2.0 – Peer relationships; PROMIS Physical Activity = PROMIS Pediatric Bank v1.0—Physical activity [45].

Discussion

The present study aimed to test the psychometric properties of the newly developed self-report scale MADRS-Y for adolescents in a normative sample.

Girls rated significantly more depressive symptoms than boys in both samples, with small effect sizes according to Cohen [48]. This difference in gender was expected, based on previous literature [68]. For age, there was a small significant difference in sample one, but not in sample two, indicating fewer depressive symptoms in the younger group of 12–15 years old, in comparison to age group 16–20 years old. This is in line with international reports [69] and literature [3].

The result from the PCA suggested that all 12 items should be used in MADRS-Y. The items loaded on the same construct of depression. Correlations between items was not too high (above .80), which supports that the items do not measure the same thing. The CFA supported the one-factor structure with good fit indices, except for RMSEA. Research has found that the model fit indices SRMR is more accurate than RMSEA when data is ordinal [70].

Measurement invariance was confirmed using Finch et al., guidelines for small and skewed data [65]. We found MADRS-Y to be equal between girls and boys, and age groups (12 to 15 years old, and 16 to 20 years old). It can be a useful measure allowing interpretation regardless of gender or age differences when screening adolescents for depression.

The reliability of the MADRS-Y scale calculated by Cronbach’s α was good, .89 for both samples separately. Test-retest reliability over three weeks was good to excellent (ICC = .87 and .91) in the current study.

Evidence of convergent and discriminant validity was shown. As expected, the MADRS-Y had the highest correlation with the MADRS-S indicating that MADRS-Y is measuring the same construct, but with a scale adapted for adolescents with easier and modernized language, and new items capturing irritability, increased appetite, and increased sleep.

The three new items in MADRS-Y

The new item ‘irritability’ showed a high item-total correlation (${\mathrm{r}}_{\text{it}}^{\mathrm{c}}$ = .6 in sample one and ${\mathrm{r}}_{\text{it}}^{\mathrm{c}}$ = .66 in sample two). This is in line with the fact that irritability is one of the core symptoms that can be observed in adolescent MDD [1,17,18,23] and is related to general symptom severity of MDD in adolescents [18].

The items that measured increased sleep and increased appetite had the lowest item-total correlation in both samples (${\mathrm{r}}_{\text{it}}^{\mathrm{c}}$ = .32–.43). However, the lowest rating was still over .3 and may therefore be included as a valid item in the self-report scale. One possible explanation for the lower ratings of increased sleep and increased appetite can be that these two items are associated with a subtype of depression described as ‘atypical depression’, in which the most important symptoms are the reverse vegetative features of overeating and oversleeping [71]. These two items were significantly correlated to each other in both samples one and two (r = .23 and r = .34) in the current study. Atypical depression is associated with a younger age of depression onset, usually during adolescence, and with concurrent anxiety, and is predominant in females [72–74]. It is important to capture these subtypes of depression for potential personalized treatment [72].

Interestingly, we found that many adolescents reported both decreased and increased sleep and decreased and increased appetite during the last three days. Correlations between decreased and increased sleep were significant and moderate (r = .41 and r = .37), and between decreased and increased appetite significant but small (r = .14 and r = .15). Adolescent sleep is typically more variable compared to children and adults [75] and eating behavior is often more irregular [76]. Clinically, sleeping disorders are common in adolescents and correlated with depressive symptoms [77]. Eating disorders are common in adolescent girls and about 12% may experience some form of bulimia nervosa or binge eating disorder [78]. Importantly, there is a strong association between bulimia nervosa and MDD in adolescent girls [79].

Compliance with DSM-5 diagnostic criteria

Including the three new items, the content of the items in MADRS-Y can be found in DSM-5 diagnostic criteria for MDD, except for the MADRS-Y item of anxiety [1]. In this current study, the anxiety item showed high item-total reliability (${\mathrm{r}}_{\text{it}}^{\mathrm{c}}$ .69 in sample one and ${\mathrm{r}}_{\text{it}}^{\mathrm{c}}$ .67 in sample two). Depressive symptoms are significantly associated with several types of anxiety in adolescents [80] and do not represent distinct constructs across development [81].

Limitations

There are several limitations in this study. A convenience sample was used, which means that generalizability is limited. The distribution of gender was skewed, which might have biased the result. Measurement invariance was calculated despite low power (100–150 in each group) however, this was corrected for, in the interpretations [65,66]. Limits on respondent burden precluded examination of monotrait-heteromethod validation. In addition, we did not investigate the adolescents’ subjective experience of filling out the MADRS-Y.

Conclusions

The CFA in the second half of the sample confirms that the factor structure of the MADRS-Y had a god fit to the model. Measurement invariance confirms that the self-report can be used independently in gender and age groups (12- to 20-year-old youth). MADRS-Y is a brief, reliable, and valid self-report questionnaire that can be used for screening for MDD among adolescents in the general population.

Future studies in clinical samples will show clinical usefulness for screening and assessment of treatment response. Furthermore, more advanced statistical methods based on modern test theory, such as item response theory, can further be used to evaluate and develop highly reliable items [82,83].

Acknowledgements

The authors would like to thank all the adolescents who participated in this project. Correspondence concerning this article should be addressed to Magnus Vestin, Department of Clinical Science, Child and Adolescent Psychiatry, Umeå University, SE-90185 Umeå, Sweden. E-mail: [email protected]

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The data that support the findings of this study are available from the corresponding author, M.V, upon reasonable request.

Additional information

Funding

This study was supported by clinical research funding (ALF) from Västerbotten county council.

Notes on contributors

Magnus Vestin

Magnus Vestin is a PhD student at Department of Clinical Science, the Child- and Adolescent Psychiatry unit, Umeå University, Sweden. He is a licensed psychologist, specialist in clinical child and youth psychology, at the Child and adolescent psychiatry clinic in Umeå.

Marie Åsberg

Marie Åsberg is an emeritus professor of psychiatry at the Karolinska institutet in Stockholm. Her research has mainly been focused on assessment and treatment of depression, the psychobiology of suicide, and more recently stress-related exhaustion disorder. She is coauthor of the Montgomery-Åsberg Depression Rating Scale (MADRS).

Marie Wiberg

Marie Wiberg is a professor in statistics with a specialty in psychometrics at Umeå University, Sweden. Her research focus on test score equating, international large-scale assessments, parametric and nonparametric item response theory, and psychometrics in general.

Eva Henje

Eva Henje is a senior specialist and associate professor in Child and Adolescent Psychiatry at Department of Clinical Science at Umeå University. Her clinical work and research is mainly focused on trauma- and stress related disorder and depression in youth.

Inga Dennhag

Inga Dennhag, PhD, psychologist and psychotherapist. She teaches and work with research at the Child and Adolescent Psychiatry at Umeå University in Sweden. Her main research areas are psychometrics, sexual harassments, teenage depression, and trauma. Year 2017, the book Power and Psychotherapy came out.