Validity and reliability of the Turkish version of the DSM-5 Separation Anxiety Disorder Severity Scale–child form

ABSTRACT

OBJECTIVE: This study aimed to assess the validity and reliability of the Turkish version of the DSM-5 Separation Anxiety Disorder Severity Scale–Child Form.

METHODS: The scale was prepared by carrying out translation and back-translation of the DSM-5 Separation Anxiety Disorder Severity Scale–Child Form. The study group consisted of 41 patients who had been treated in a child psychiatry unit and diagnosed with separation anxiety disorder and 100 healthy volunteers who were attending middle or high school during the study period. For the assessment, Screen for Childhood Anxiety and Related Emotional Disorders (SCARED) was also used, along with the DSM-5 Separation Anxiety Disorder Severity Scale–Child Form.

RESULTS: The Cronbach alpha internal consistency coefficient was calculated as 0.932, while the item–total score correlation coefficients were between 0.400 and 0.874. One factor that could explain 63% of the variance was obtained. The scale showed a medium correlation with SCARED. The area under the receiver operating characteristic curve was calculated as 0.898.

CONCLUSION: It was concluded that the Turkish version of DSM-5 Separation Anxiety Disorder Severity Scale–Child Form could be used as a valid and reliable tool both in clinical practice and for research purposes.

KEYWORDS:

• DSM-5
• DSM-5 Separation Anxiety Disorder Severity Scale
• reliability
• validity
• anxiety disorder

Introduction

Separation anxiety disorder (SAD) is an intense and extreme anxiety and fear that is felt by an individual within the context of him/her separating from the people he/she is emotionally attached to, which is developmentally inappropriate and lasts for at least 4 weeks. It is common that the child feels he/she will lose his/her main attachment figures, an overt, extreme, and continuous fear of experiencing some trouble or a situation that might trigger an unexpected and undesired separation from these figures, unwillingness to attend school or to go to other places due to fear of separation, avoidance of being alone at home or outside or being in situations where he/she has to be separate from his/her main attachment figures, continuous nightmares about the issue of separation, and a reluctance to go to sleep without one of his/her main attachment figures present at his/her side. Somatic symptoms frequently accompany the clinical course within the context of separation [1].

The frequency of SAD in children is approximately 4–5% [2]. Long-term follow-up studies reported that children who refuse to attend school due to SAD and are treated for the condition experience ongoing emotional difficulties even though their school adjustment improves over time [3]. Recent studies suggested that SAD in childhood is a risk factor for other anxiety disorders and depression during adulthood [4], making this an important condition that needs to be recognized and treated early.

The diagnostic assessment of SAD is performed mainly through a clinical interview. One needs to focus on the onset and development of anxiety symptoms, the severity of symptoms, effects of symptoms on functioning, and stressors related to symptoms during the interview [5]. Although the most commonly used method for evaluation is a clinical interview, some self-report scales to aid in the assessment of the clinical picture, such as the Multidimensional Anxiety Scale for Children (MASC) and SCARED, have been presented in the literature [2]. However, these scales are not specific to SAD. Indeed, they interrogate the presence of any anxiety disorder according to DSM-IV criteria, resulting in an overall assessment. Although they were confirmed to be effective in distinguishing children with anxiety disorders and those without, it has been largely ignored that age-appropriate and disorder-specific scales are much needed during both diagnosis and follow-up [6,7]. Separation Anxiety Scale for Children and Separation Anxiety Assessment Scale are among the specific scales used within this field [8]. However, these instruments lack Turkish validity and reliability as they have not been studied so far.

The most commonly used diagnostic system for the classification of illnesses and disorders in psychiatry, the Diagnostic and Statistical Manual of Mental Disorders, is prepared and renewed within certain intervals by the American Psychiatric Association and has been published in its 5th edition [1]. With publication of the DSM-5 Diagnostic Criteria handbook in 2013, novel instruments and tools were warranted to determine severity and monitor follow-up of disorders in both daily psychiatry practice and field studies, resulting in the recommendation of novel assessment tools that have been adjusted to DSM-5 criteria for many psychiatric disorders [9].

The DSM-5 Separation Anxiety Disorder Severity Scale–Child Form is an instrument that determines the severity of SAD symptoms in children and adolescents 11–17 years of age. The scale was designed to be used in the initial assessment and treatment processes of children and adolescents diagnosed with SAD (or cases that have clinically severe SAD symptoms) [9].

The basic reason why we attempted to translate and validate the measure was because we need a specific scale that assesses the SAD entity. Although this disorder is commonly seen in clinical practice in our country, there have not been many large-scale studies regarding it, and nationwide epidemiological studies are even more scarce. In one recent study attempting to determine the prevalence of the disorder in Izmir, a city in Turkey, the prevalence rate was found to be 1% among primary schoolers [10]. In Turkey, two other studies in this field included participants aged 7–12 years. In both studies, the most common complaint at the time of application to a child psychiatry unit was refusal to attend school, and high rates of comorbidity with other anxiety disorders were seen [11,12]. To the best of our knowledge, no research has been conducted on SAD in adolescents in Turkey. Not having specific tools designed to measure the symptoms of this disorder in adolescents might be one of the reasons for the shortage of studies in the relevant age group. In that sense, our study and adaptation of a tool that could be used in this age group might accelerate novel studies in adolescents with SAD. The use of this tool might provide standardization with other studies worldwide and might also lead to the scale being used as a screening instrument.

The aims of this study were to prepare the Turkish version of the DSM-5 Separation Anxiety Severity Scale to test its psychometric properties in Turkish populations, to determine its cultural adaptability through a translation/back-translation process, to determine the internal consistency coefficients of the DSM-5 Separation Anxiety Disorder Severity Scale, to evaluate its construct validity by performing factor analysis, to determine correlations with SCARED (criterion validity), and to assess its discriminative validity among clinical and nonclinical samples.

Methods

Translation process

With the intention of carrying out the Turkish adaptation of the DSM-5 Separation Anxiety Disorder Severity Scale, written permission was obtained from HYB Publishing Firm and Boylam Psychiatric Institute regarding approval to initiate scale studies, as they held the translation and publication rights of the DSM-5 Source Book and Handbook. Recommendations of the World Health Organization were applied throughout the translation and adaptation process of the scale [13]. The translation was performed by two experienced specialists working within the field of child and adolescent psychiatry and an adult psychiatrist who knew the English language well, independent of each other. The final draft of the manuscript was organized with participation of both the translation and the research teams. Translation was controlled and was turned into a text regarding meaningfulness, use of language, and cultural, conceptual, and writing appropriateness with the intent of emphasizing conceptual rather than literal translations as well as the need to use natural and acceptable language for the broadest audience. Then, the scale was translated back into English by another adult psychiatrist who knew the English language well who was also blind to any information in relation to the scale. This final translation material was compared to the original format of the scale by the whole team with regard to its ability to meet the concepts it covers. To test the readability of the translated measure, the scale was applied to 10 children (5 girls and 5 boys) who had been diagnosed with an anxiety disorder and had sought help from a child psychiatry unit, prior to a clinical interview. Conceptual penetrability and comprehension of the scale were assessed within the clinical interview. There was no question that was regarded as lacking clarity or not being understood by the study populations.

Sample group

The study subjects consisted of healthy volunteers and patients seen in the Celal Bayar University Child Psychiatry Outpatient Unit. The community sample that represented the low-psychiatric-risk group was obtained from schools in the area. Discrepancies regarding criteria and perspective over the required sample size to conduct multivariate analyses for assessing psychometric properties and adaptation of scales, such as factor analysis, are present in the relevant literature. One of the common suggestions on this matter is that the sample size needs to be 5 or even 10 times the number of items [14]. To reach an appropriate sample size based on basic research statistics, the nonclinical sample size was calculated as 100 in our study. Inclusion criteria for the control group comprised being between 11 and 17 years of age, with no reported history of mental health problems or physical disorders, and having enough intellectual capacity to follow the study instructions. Three patients from the nonclinical sample were excluded due to physical illnesses and one was also excluded for having a psychiatric disorder.

To determine whether the scale could discriminate clinical and nonclinical samples, the clinical sample was established, which represented a high-risk group regarding psychiatric disorders including 41 adolescents between 11 and 17 years of age diagnosed with SAD according to DSM-5 criteria who had attended the child and adolescent psychiatry unit at CBU Medical School Hospital. Children who had consulted with the child psychiatry unit and had been evaluated and formally diagnosed by any child psychiatrist in the unit were referred to the research team after giving information to both the child and the parents and obtaining their consent. Diagnoses in the patient group were made using clinical interviews based on the DSM-5 diagnostic classification system. Fourteen of the patients (34.1%) had just been diagnosed at the time of the study and had not been treated, while 27 cases (65.9%) were being followed up in the child psychiatry unit (for 1–6 months) and were being treated. Inclusion criteria comprised being 11–17 years of age, meeting the criteria for SAD according to the DSM-5, and having a high enough level of intellectual functioning to follow the study instructions. Exclusion criteria included having a physical or neurological disorder that would require continuous treatment. Ethical approval for this study was granted by Celal Bayar University Medical School Clinical Research Evaluation Committee.

Assessment tools

1. Sociodemographic data form: The sociodemographic data form was created by the researchers to collect such data from the study group. The form included questions on age, gender, school attendance, whether the subject had a physical or a psychiatric disorder, age of the parents, educational levels of the parents, whether the parents work, the structure of the family, diagnosis of the case, time since diagnosis, and treatment-related issues.

2. DSM-5 Separation Anxiety Disorder Severity Scale–Child Form: The DSM-5 Separation Anxiety Disorder Severity Scale–Child Form is a scale with 10 items that determines the severity of SAD symptoms in children and adolescents between 11 and 17 years old. The scale was designed to be used in the first assessment and for the follow-up of children and adolescents with SAD (or with severe clinical separation anxiety symptoms). It involves a self-report form for the child to fill out. For each item, the individual is asked to rate the severity of separation anxiety symptoms in the last 7 days. Each item is rated on a five-point scale, ranging from zero to four (0 = never; 1 = occasionally; 2 = half of the time; 3 = most of the time; 4 = all the time). Total score varies between 0 and 40, with higher scores reflecting a more severe form of the clinical condition. In a study conducted in the Netherlands within a community sample that included children between 8 and 13 years old, the scale was found to be valid and reliable [15].

3. Screen for Child Anxiety and Related Disorders–SCARED: The Screen for Child Anxiety and Related Disorders–SCARED was developed by Birmaher et al. (1999) to screen childhood anxiety disorders [16]. A Turkish validity and reliability study was performed by Çakmakçı (2004), and the scale has both parent and child forms [17]. Although the time frame referred to in the questions is the past 3 months, SCARED can also be used to evaluate current or lifetime anxiety disorders. SCARED comprises 41 items that are rated on a three-point scale of responses (0 = not true, 1 = sometimes true, and 2 = often true). The scale also has subscales for somatic/panic, generalized anxiety, separation anxiety, social anxiety, and school phobia symptoms. In addition, the total anxiety score is the simple sum of all items. A score of 5 for items 4, 8, 13, 16, 20, 25, 29, and 31 might indicate the presence of SAD. In our study, we used the child form of SCARED to assess the DSM-5 Separation Anxiety Disorder Severity Scale and its concurrent validity.

Plan and schedule of the study

The data collection phase of this study was between May 2015 and August 2016. Cases consulting within this period, meeting the inclusion criteria, and volunteering to be included were briefed about the study after being included in it. Exclusion criteria were evaluated and a written informed consent form was collected from the participants and their parents. The sociodemographic data form was filled out by each participant in the study. Meanwhile, for validity assessment, each participant was asked to fill out the DSM-5 Separation Anxiety Scale–Child Form. For validity, both DSM-5 Separation Anxiety Disorder and SCARED were applied, and responses were provided by the children themselves. Clinical interviews based on the DSM-5 diagnostic classification were applied to cases within the clinical sample. No clinical interviews were conducted for cases in the nonclinical sample; however, a sociodemographic data form that assessed the presence of any physical or psychiatric disorder completed by the children and also their caregivers was evaluated.

Statistical analysis

The study data were analysed using Statistical Package for Social Sciences (SPSS) 15.0 for Windows and AMOS 18.0 statistical package programs. The chi-square test was used to analyse the difference between categorical variables. Student’s t-test was used to analyse the difference between continuous variables of the two groups. Pearson’s and Spearman’s correlation coefficient tests were used to examine the relationship between parameters. A two-tailed p-value of .05 was set as the cut-off for statistical significance. To control for the effects of differences in age and gender between the clinical and nonclinical samples on the total scale scores, ANCOVA was applied [18].

A Cronbach alpha internal reliability analysis was also performed, and the item–total score correlation coefficients were measured to determine the reliability of the scale [19]. For the construct validity of the scale, both exploratory and confirmatory factor analyses were performed using all data of the study groups. First, to control the congruity of the sample for the exploratory factor analysis, the Kaiser–Meyer–Olkin and Bartlett tests were used. The exploratory factor analysis was carried out by applying varimax rotation based on the principal components method, and factors with eigenvalues equal to 1 or greater were included. Among the factor constructs, items with factor loads of 0.4 and above were included in the analysis. The exploratory factor construct was compared to the original dimension structure of the scale. Even though this was a single-factor scale, to obtain an understanding of whether the Turkish form was a similar construct or not (to elaborate the construct over Turkish communities) and to obtain more information about the nature of factors measured by the instrument, exploratory factor analysis was implemented. The results showed that it was congruent with the original construct. Rotation was not applied as only a single factor was obtained. The consistency model of data and its adjustment to the model were assessed. Different types of goodness-of-fit indices were used [root mean square error of approximation (RMSEA) and comparative fit index (CFI)]. Values of RMSEA below 0.05 indicate good adjustment with the data, 0.05–0.08 acceptable adjustment, 0.08–0.1 poor adjustment, and greater than 0.1 unacceptable adjustment. CFI values can vary between 0 and 1, but should be greater than 0.9 [20].

The strength for distinguishing between community and clinical samples was shown by the receiver operating characteristic (ROC) curve. Measures of the area under the ROC curve equal to or above 0.9 indicate good discriminative strength, while measures between 0.8 and 0.9 are regarded as acceptable [19]. In addition, correlation between the Separation Anxiety Disorder Severity Scale–Child Form and SCARED was assessed for concurrent validity. For validity and reliability analyses, data from 141 children were used.

Results

The study included 41 patients consulting at the CBÜ Child Psychiatry Unit who had SAD and 100 healthy volunteers. The data were normally distributed. The sociodemographic and clinical features of the study groups are shown in Table 1. Table 2 presents mean total scores of the applied scales by gender.

Table 1. Sociodemographic data of the study groups.

		Separation anxiety disorder group N:41	Control group N:100
Age*		14.0 ± 2.2	15.7 ± 1.1
Gender*	Female	27(65.9%)	42(42%)
	Male	14(34.1%)	58(58%)
School	Attending	41(100%)	100(100%)
	Not attending	0	0
Time since diagnosis	New diagnosis	14(34.1%)
	1–6 months	27(65.9%)
	More than 6 months	0(0.0%)
Medication use	Antidepressants	27(65.9%)
	Antipsychotics	7(17.1%)
	Benzodiazepines	0
DSM-5 Separation Anxiety Disorder Scale–Child Form*		20.5 ± 7.6	5.9 ± 8.1
Separation anxiety subscale of SCARED*		9.9 ± 3.3	3.4 ± 3.1
SCARED total score*		44.6 ± 12.8	21.9 ± 13.9

*p < .05.

Table 2. Mean total scores of applied scales by gender.

	Total sample N = 141 Mean ± SD	Female N = 69 Mean ± SD	Male N = 72 Mean ± SD
DSM-5 Separation Anxiety Disorder Scale–Child Form*	10.1 ± 10.3	12.9 ± 10.6	7.5 ± 9.4
Separation anxiety subscale of SCARED*	5.2 ± 4.3	6.7 ± 4.3	3.9 ± 3.9
SCARED total score*	28.2 ± 17.0	34.1 ± 16.4	22.7 ± 15.8

*p < .01.

Mean age and gender ratio differed between the clinical and nonclinical samples. To control the effect of this difference on the total scale scores, ANCOVA was applied. In the model, gender and group were regarded as fixed factors, while age was a covariate. When the effects of age and gender were controlled, the DSM 5 Separation Anxiety Scale–Child Form total scores remained significantly different between the two groups (clinical and nonclinical samples) (F = 82.823; p = .0001).

Twenty-seven of the 41 patients (65.9%) who formed the clinical sample had ongoing treatment. Fourteen (34.1%) were newly diagnosed. Among the cases, 85.4% (n = 35) had a comorbid diagnosis. A total of 48.8% (n = 20) had social anxiety disorder, 24.4% (n = 10) had generalized anxiety disorder, 17.1% (n = 7) had specific phobia and panic disorder, 12.2% (n = 5) had agoraphobia, 9.8% (n = 4) had attention-deficit hyperactivity disorder, 4.9% (n = 2) had eating disorders, and 2.4% (n = 1) had depressive disorder.

Reliability analyses

The Cronbach alpha internal consistency coefficient was determined to be 0.932 in the reliability analyses of the DSM-5 Separation Anxiety Disorder Severity Scale–Child Form. Cronbach’s alpha with each item extracted is shown in Table 3. The item–total score correlation coefficients were found to be between 0.400 and 0.874 (Table 3).

Table3. Item–total score correlation coefficients, Cronbach’s alpha coefficients, and factor loads for the items in the DSM-5 Separation Anxiety Disorder Severity Scale–Child Form.

	Item–total score correlation coefficients	Itemless Cronbach’s alpha coefficients	Factor loads
Felt moments of sudden terror, fear, or fright when separated	0.819	0.920	0.868
Felt anxious, worried, or nervous about being separated	0.773	0.923	0.828
Had thoughts of bad things happening to people important to me or bad things happening to me when separated from them (e.g. getting lost, accidents)	0.805	0.921	0.853
Felt a racing heart, sweaty, trouble breathing, faint, or shaky when separated	0.732	0.925	0.789
Felt tense muscles, felt on edge or restless, or had trouble relaxing or trouble sleeping when separated	0.650	0.930	0.714
Avoided going places where I would be separated	0.874	0.918	0.905
When separated, left places early to go home	0.751	0.924	0.808
Spent a lot of time preparing for how to deal with separation	0.788	0.923	0.836
Distracted myself to avoid thinking about being separated	0.728	0.925	0.786
Needed help to cope with separation (e.g. alcohol or medications, superstitious objects)	0.400	0.939	0.456

Validity analyses

To determine the construct validity, an exploratory factor analysis was used to assess the DSM-5 Separation Anxiety Disorder Severity Scale–Child Form. Before the exploratory factor analysis, a Kaiser–Meyer–Olkin analysis was used to assess whether the sample was in congruity, and the results showed a coefficient value of 0.924. Using the Bartlett test, the chi-square value was calculated to be 1026.385 (p < .0001). These results indicate that the sample group was congruent with the factor analysis.

In the factor analysis, one factor with an eigenvalue greater than 1 was confirmed, which explained 63% of the total variance (Table 3). All factors of the scale were represented within the factor construct. The eigenvalue of the factor was 6.296, which explained 63% of the total variance. Factor loads are shown in Table 3.

To determine the congruity of the scale construct, confirmatory factor analysis was performed and the distribution of the sample was assessed. The CFI value for the model constructed according to the single dimension construct of the scale was 0.932, while the RMSEA value was 0.10, as determined by confirmatory factor analysis (Table 4).

Table 4. Single-factor confirmatory factor analysis results of DSM-V Separation Anxiety Disorder Severity Scale–Child Form.

Adjustment tests	Adjustment values of DSM-V Separation Anxiety Disorder Severity Scale–Child Form
CFI	0.932
NFI	0.902
RMSEA	0.10

Note: CFI: comparative fit index; NFI: normed fit index; RMSEA: Root mean square error of approximation.

With the ROC analysis of the DSM-5 Separation Anxiety Disorder Severity Scale–Child Form that included the healthy group and the group diagnosed through a clinical interview, the area under the ROC curve was measured at 0.898. The scale has a cut-off score of 13 and scores equal to 13 and above had a sensitivity of 80.5% and specificity of 83%.

In the concurrent validity analysis for the DSM-5 Separation Anxiety Disorder Severity Scale–Child Form and SCARED, the correlation coefficients were found to be r = .682, p < .0001, for the total score of the scale and r = .636, p < .0001, for the separation anxiety subscale.

Discussion

This study examined the validity and reliability of the Turkish version of the DSM-5 Separation Anxiety Disorder Severity Scale–Child Form and showed that the Turkish version is acceptable. The initial reliability study of the scale in children yielded a Cronbach alpha correlation coefficient of 0.86 [15]. For psychometric assessments, the closer the Cronbach alpha coefficient value gets to 1, the higher the reliability of the scale [19]. In our study, the internal consistency of the Cronbach alpha coefficient was 0.932, indicating high reliability, which reflects that the scale construct can appropriately represent the whole. Item–total score correlation coefficients were found to be high again, proving the reliability of the scale construct.

In concurrent validity analysis, correlation with SCARED was measured. In the first study of the scale measured in children, correlation with SCARED was assessed and the coefficient was found to be 0.48 [15]. In our study, the scale had a medium correlation with the separation anxiety subscale of SCARED (r = .636) and again a medium correlation with the total anxiety scores of SCARED (r = .682); both were statistically significant. Among the cases included in this study, 85.4% had a comorbid disorder. Moreover, 65.9% of the cases in the clinical sample involved patients who had started treatment and were under follow-up. Cases might have scored higher on a scale that assesses all anxiety disorders, such as SCARED. Total SCARED scores also had a higher correlation with DSM-5 Separation Anxiety Disorder Severity Scale scores, compared to the correlation between SCARED’s SAD subscale and DSM-5 Separation Anxiety Disorder Severity Scale scores. The questions in the DSM-5 Separation Anxiety Disorder Severity Scale also refer to avoidance behaviour as well as cognitive and physical symptoms related to fear and worry. The number of items that assess the presence of avoidance behaviour in SCARED is small. As 65.9% of cases in the clinical group were being treated at the time, the rate of avoidance behaviour might have decreased, resulting in lower scores. This might explain the lower correlation coefficient measured in concurrent validity analyses and, again, might account for the higher correlation coefficient with total scores due to the presence of somatic symptoms within other subscales of SCARED. Concurrent validity of the scale indicates that the scale could be used as a valid instrument.

An exploratory factor analysis was used to assess the DSM-5 Separation Anxiety Disorder Severity Scale. In our study, one factor with an eigenvalue over 1 was obtained. Specifically, the eigenvalue of this factor was 6.296; it explained 63.0% of the total variance and was in congruity with the original scale construct [15]. The second tool applied to test construct validity was confirmatory factor analysis, in which the dimensional model structure of the scale and its adjustment were assessed. The findings showed that the scale construct was appropriate with the model.

The scale could correctly assess clinically positive cases diagnosed with SAD upon clinical assessment at a rate of 80.5%. The scale could also successfully discriminate individuals without the disorder determined through clinical assessment at a rate of 83%. As is well known, higher sensitivity might indicate that the nonclinical group could be well distinguished from the pathological subjects, while higher specificity indicated that it would be appropriate in discriminating normal cases. Sensitivity and specificity of the DSM-5 Separation Anxiety Disorder Severity Scale–Child Form were both high. This might be interpreted as meaning that the scale could be used both for screening and for clinical follow-up.

Both the construct and concurrent validities indicate the validity of the Turkish version of the DSM-5 Separation Anxiety Disorder Severity Scale–Child Form. The primary limitation of this study is that the subjects in the clinical group had comorbid diagnoses with SAD and other anxiety disorders. It is important to carry out additional studies to determine features of the scale that discriminate SAD from other psychiatric disorders. The presence in the clinical sample of subjects who were newly diagnosed and untreated as well as patients who had been treated could also be considered as another limitation. Associated with this, the absence of children with longstanding SAD might also be a limitation. Two other limitations were that we did not use a structured clinical diagnostic interview to search for possible diagnoses in the control group and we did not perform test–retest procedures. In addition, the significant differences between the patient and control groups regarding age and gender must be taken into consideration when evaluating the differences obtained in the comparative analyses of the items in the scale. In directing the study aims, most of the statistical analyses could be accomplished without excluding any subjects with the given sample size. The strength of this study is that the sample might be representative of patients in clinical practice.

Our results confirmed that the scale is a valid and reliable instrument, and strongly suggest that the scale could be used in future SAD studies. This study adds a novel scale that could be used in research to the literature. The scale was also shown to be appropriate for clinical studies, treatment follow-up, monitoring, and trauma, and could be widely used.

Informed consent

Informed written consent was obtained from all individual participants included in the study.

Research involving human participants and/or animals

All procedures performed in studies involving human participants were in accordance with the ethical standards of institutional and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Disclosure statement

No potential conflict of interest was reported by the authors.