ESSENCE-Q: Slavic language versions for developmental screening in young children

Abstract

Purpose

The Early Symptomatic Syndromes Eliciting Neurodevelopmental Clinical Examinations – Questionnaire (ESSENCE-Q) was developed as a brief screener to identify children with developmental concerns who might have neurodevelopmental disorders (NDDs). This study aimed to translate the ESSENCE-Q into south Slavic languages, namely, Bosnian, Bulgarian, Croatian, Macedonian, Montenegrin, Serbian, and Slovenian, and to evaluate its psychometric properties for screening purposes in clinical settings.

Patients and methods

In the study, the ESSENCE-Q was completed for 251 “typically developing” children and 200 children with 1 or more diagnosed NDDs, all aged 1–6 years. Internal consistency and construct validity were tested first, followed by generating receiver operating characteristic curves and the area under the curve. Optimal cutoff values were then explored.

Results

The Cronbach’s α coefficients were 0.91, 0.88, and 0.86 for ESSENCE-Q parent-completed form, and the telephone and direct interview forms administered by trained nurse or specialist, respectively. The 3 versions produced area under the curve values (95% confidence interval): 0.96 (0.93–0.99), 0.91 (0.86–0.95), and 0.91 (0.86–0.97), respectively. An optimal cutoff for ESSENCE-Q parent-completed form was found to be ≥3 points, while for the telephone and direct interviews, it was ≥5 points.

Conclusion

We found adequate measurement properties of the south Slavic languages versions of the ESSENCE-Q as a screener for NDDs in clinical settings. This study provided additional data supporting sound psychometric properties of the ESSENCE-Q.

Keywords:

• early screening
• questionnaire
• autism
• intellectual disability

Introduction

Neurodevelopmental disorders (NDDs) are a group of conditions originating early in life with a wide range of developmental deficits.¹ Roughly estimated, the cumulative prevalence of all NNDs is about 10% in children and adolescents <18 years of age.^1,2 The deficits are present in 1 or more emotional, cognitive, behavioral, motor, and/or social communication developmental aspects, with a range of impairments in personal, social, and, later on, academic, and/or occupational functioning. According to the most recent classification, there are clearly separated NDDs, such as intellectual disability/intellectual developmental disorder (IDD), communication disorders, autism spectrum disorder (ASD), attention-deficit/hyperactivity disorder (ADHD), neurodevelopmental motor disorders, and specific learning disorders.¹ Due to common neurobiological and genetic etiologies, NDDs frequently co-occur, overlap, and/or precede one another, bringing a huge challenge for proper diagnosis and treatment.³

Evidence indicates that early identification and treatment initiation have beneficial developmental effects and lead to favorable outcomes in children with NDDs.^4–7 In this regard, early screening is of significant importance and many different screening questionnaires have been developed over the past 2 decades.^8–10 However, from clinical and research point of view, there are few challenges to consider when screening with available questionnaires.^2,11,12 Available screening questionnaires generally categorize the child as having or not having 1 particular NDD based on the presence of specific symptoms/deficits related to that disorder, while mainly ignoring the presence of other developmental deficits, which could also categorize the child as having another or additional NDD. Related to this, a majority of these questionnaires is focusing on 1 or 2 NDDs only, which although maximizing the sensitivity and specificity for selecting 1 specific disorder during screening, does not reflect the real-world circumstances in which NDDs are co-occurring and overlapping in young children.¹ In addition, when screening for a particular disorder, it is required to take into account early developmental aspects and the course of the disorder screened, which creates problems because NDDs present differently at different ages with symptoms changing markedly, which may also affect the results of the screening.¹³ Finally, specific screening questionnaires may have different performances when used for clinical- and community-based screening, with respect to wide ranges of symptoms and severity levels present in one or the other.

ESSENCE is an acronym for Early Symptomatic Syndromes Eliciting Neurodevelopmental Clinical Examinations referring to neurodevelopmental or neuropsychiatric disorders including IDD, ASD, ADHD, oppositional defiant disorder (ODD), tic disorders, developmental coordination disorder (DCD), speech and language disorder (SLD), and others.² The concept was introduced with an idea to alert clinicians and researchers of the complexity and overlap of neurodevelopmental problems with early onset and to act toward diagnosis and treatments accordingly without putting children into mere “diagnostic boxes”.² The ESSENCE-Questionnaire (ESSENCE-Q) was developed from the mentioned concept as a brief screener to identify children who may have neurodevelopmental difficulties, namely, difficulties in general development, communication/language, social interrelatedness, perception, motor coordination, attention, activity, behavior, mood, feeding, and/or sleeping in the first years of childhood, which could all be seen as red flags for possible NDDs.¹⁴ The ESSENCE-Q is intended to be used in the form of a brief interview by a specialist/nurse or as a questionnaire to be completed by parents and/or caregivers. Those children who are identified as raising concern regarding development are referred for detailed diagnostic evaluation. Two psychometric studies of the questionnaire have been performed in Japan, one in a neurodevelopmental clinic and one in a child health center. These studies have demonstrated good internal consistency, sensitivity, and specificity for screening purposes, when it was used as a parent-completed and direct (ie, face-to-face) interview.^12,15

The ESSENCE-Q could overcome the aforementioned challenges when screening with available questionnaires, particularly in detecting children with broader neurodevelopmental difficulties, but not only specific NDDs, and could be used in different settings, either clinical or non-clinical, which is especially relevant in low-resource settings and low-middle income countries.¹⁶ In order to use the ESSENCE-Q as a gold standard for NDDs screening, more psychometric studies with different forms and language versions and in different clinical and non-clinical settings are needed.

The present study was organized with the aims to 1) translate the ESSENCE-Q into 7 south Slavic languages, namely, Bosnian, Bulgarian, Croatian, Macedonian, Montenegrin, Serbian, and Slovenian and 2) evaluate its psychometric properties for screening purposes in clinical settings. Three ESSENCE-Q forms were considered; a parent form for self-completion and telephone interview and direct interview administered by a specialist or a nurse.

Patients and methods

This is a clinic-based, multicenter validation study. The Ethics Committee of the Clinic for Neurology and Psychiatry for Children and Youth, Belgrade, Serbia, first approved the study and afterward, it was also approved by the relevant bodies of each included clinic where participants were recruited.

ESSENCE-Q

The ESSENCE-Q has 12 items covering concerns related to child’s 1) general development, 2) motor development/milestones, 3) sensory reactions (eg, touch, sound, light, smell, taste, heat, cold, and pain), 4) communication/language/babble, 5) activity (overactivity/passivity) or impulsivity, 6) attention/concentration/“listening”, 7) social interaction/interest in other children, 8) behavior (eg, repetitive and routine insistence), 9) mood (depressed, elated/manic, extreme irritability, and crying spells), 10) sleeping, 11) feeding, and 12) “funny spells”/absences. Response options are “Yes” (2 points), “Maybe/A little” (1 point), or “No” (0 point) for each item. Total score is a sum of all answered items (possible range 0–24).

After receiving the author’s approval (C Gillberg), the ESSENCE-Q was translated into the South Slavic group of languages (ie, Bosnian, Bulgarian, Croatian, Macedonian, Montenegrin, Serbian, and Slovenian). South Slavic languages are considered to constitute a dialect continuum in a set of phonological, morphological, and lexical innovations (isoglosses) spoken mainly in Balkan countries, and also in other European countries.¹⁷ The members of the research team conducted the translation process separately for each language following the same procedure¹⁸ to ensure content equivalence with the original. The translation process included: forward translation (2 independent forward translations), reconciliation (1 single translation developed from the 2 forward translations), backward translation (1 independent back-translation), harmonization (all forward and backward translations analyzed together and a pre-final version for a pilot testing developed), and pilot testing phase (several independent semi-structured interviews held with parents of children with NDDs). For all developed versions, there was no need to culturally adapt any item, considering that the terms used to describe developmental aspects are the same or very similar, common in the medical literature, and there are no cultural differences between the neighbor countries included. During the pilot testing phase, the instructions and 10 items were estimated as comprehensive, precise, and relevant for assessing developmental and emotional/behavioral difficulties in young children, thus they remained unchanged and no terms were added, replaced, or omitted. Nevertheless, in Item 2 (motor development/milestones), “milestones” was omitted, considering that most parents did not understand this word. In Item 12 (“funny spells”/absences), the slang “funny spells”, referring to an epileptic seizure, was not possible to translate accordingly with a slang. Therefore, in some versions, the translation was adapted as “looking weird for short periods of time” or “atypical behavior”, while “absence” was translated as “periods when one is looking absent” (the back-translation for this item is “Short periods when one is absent/looking lost for short periods of time”), with the original meaning preserved across the versions.

Participants and procedures

The participants in the present study were parents/caregivers of children referred for diagnostic and/or therapeutic purposes during January 2017–July 2017 to one of the following clinics: Clinic for Neurology and Psychiatry for Children and Adolescents Belgrade, Serbia; University Children’s Hospital Skopje, Macedonia; Department of Child Psychiatry, University Clinical Center Ljubljana, Slovenia; Department of Child and Adolescent Psychiatry, University Hospital St Marina Varna, Bulgaria; Pediatric Clinic, Clinical Center Sarajevo, Bosnia and Herzegovina; Mental Health Center Pljevlja, Montenegro; and Clinic for Children’s Disorders, Clinical Hospital Center Split, Croatia. In each clinic, the team included a child psychiatrist, pediatric neurologist/developmental and behavioral pediatrician, psychologist/speech therapist/IDD specialist, and neurodevelopmental nurse.

The main inclusion criterion for the study was a child’s age between 12 months and 6 years. Children were selected from the aforementioned centers through convenience sampling and all were separated into 2 groups; children with NDDs – 1) ESSENCE group and children with normal developmental trajectory – 2) non-ESSENCE group. A child was placed into the ESSENCE group if diagnosed with ASD, ADHD, ODD, tic disorders, DCD, SLD, IDD, global developmental delay, borderline intellectual functioning, behavioral phenotype syndromes, and/or neurological and seizures presenting with major behavioral/cognitive problems. The non-ESSENCE group consisted of children with age-expected neurodevelopment and no ESSENCE diagnosis, but who might have had unspecific and unrelated symptoms to NDDs, such as feeding, sleeping, or attachment problems, separation anxiety, and/or similar. The first group was directly available to the teams of the study, while the non-ESSENCE group was derived from 3 sources. The first source was a pediatric department, where children were referred for physical disorders, but for whom was sought a psychological/psychiatric consultation from the team members. The second source was the database of children included in an early autism screening program in Serbia.¹⁹ The third source included children assessed for NDDs during the study period to which any neurological or psychiatric disorder was excluded. All child psychiatric/NDDs were diagnosed using the Diagnostic and Statistical Manual of Mental Disorders – 5¹ and/or International Statistical Classification of Diseases – 10.²⁰ Children were categorized into the 2 groups based on an established clinical protocol for a diagnostic assessment of each included center, including psychological and development assessment procedures, whereas there was no common method (ie, psychological and diagnostic scales) to implement at the beginning of the study and use across all centers.

Parents of selected children were informed about the study and those who agreed to participate and provided a written consent were included. The ESSENCE-Q was administered to parents as a self-complete form (ie, parent-completed form) and as a telephone or direct interview by trained nurse/specialist (ie, telephone or direct interview form). In the case of the children who were assessed before the study, mostly telephone interviewing was organized. For the children assessed for NDDs during the study, the parent either self-completed the form or direct interviewing was organized. Each of the 3 assessments was organized and supervised by the same specialist or nurse blinded for the health status of the child. The responsible person was instructed only to supervise a parent self-completing and/or to conduct the interviews. It was allowed only to discuss the ESSENCE-Q items with the parents, without interfering with the responding or discussing the diagnosis of the child. Parent self-reporting or direct interviewing were organized during the diagnostic work-up of the child.

Over the study period, complete data were collected from 451 children; 37 (8.2%) from Bosnia and Herzegovina, 50 (11.1%) from Bulgaria, 8 (1.8%) from Croatia, 109 (24.3%) from Macedonia, 16 (3.5%) from Montenegro, 148 (32.8%) from Serbia, and 83 (18.4%) from Slovenia. The non-ESSENCE group included 251 children (128 [51%] boys; mean age 3.1 years), of whom 45 had some psychological symptoms not related to NDDs (15 [33.3%] fears/anxieties, 10 [22.2%] sleeping problems, and 20 [44.4%] other symptoms). The ESSENCE group included 200 children with 1 or more NDDs (142 [71%] boys; mean age 3.9 [1.2] years; Table 1).

Table 1 Diagnoses of 200 children in the ESSE NCE group

ESSENCE diagnosis	N (%)
Attention deficit/hyperactivity disorder	12 (6.0)
Attention deficit/hyperactivity disorder and tics	1 (0.5)
Autism spectrum disorder	37 (18.5)
Autism spectrum disorder and intellectual developmental disorder	30 (15.0)
Attention deficit/hyperactivity disorder and intellectual developmental disorder	2 (1.0)
Behavioral syndrome	10 (5.0)
Borderline intellectual functioning	4 (2.0)
Epilepsy	11 (5.5)
Epilepsy and speech and/or language disorder	3 (1.5)
Global developmental delay	21 (10.5)
Intellectual developmental disorder	31 (15.5)
Intellectual developmental disorder and motor abnormality	9 (4.5)
Intellectual developmental disorder and epilepsy	3 (1.5)
Motor disorder	10 (5.0)
Oppositional defiant disorder	4 (2.0)
Speech and/or language disorder	9 (4.5)
Speech and/or language disorder and attention deficit/hyperactivity disorder	3 (1.5)

Abbreviation: ESSENCE, Early Symptomatic Syndromes Eliciting Neurodevelopmental Clinical Examinations.

In total, 147 ESSENCE-Q parent-completed forms were available; 48 (32.7%) for the ESSENCE and 99 (67.3%) for the non-ESSENCE group. ESSENCE-Q telephone interview forms were available for 188 children; 88 (46.8%) children in the ESSENCE and 100 (53.2%) in the non-ESSENCE group. ESSENCE-Q direct interview forms were available for 116 children; 52 (44.8%) children in the ESSENCE and 64 (55.2%) in the non-ESSENCE group. Age and gender distributions of the participants in the ESSENCE and non-ESSENCE group are given in Table 2. For the parent-completed and telephone interview forms, data were provided for significantly older children in the ESSENCE, than in the non-ESSENCE group (P<0.05). For all 3 forms, more data were provided from male children in the ESSENCE, than in the non-ESSENCE group (P<0.02). Mothers mostly provided data (83.4%), followed by fathers (14.6%) and significant others (1.9%).

Table 2 Age and gender distributions of the ESSENCE and non-ESSENCE groups

Age and gender distribution	Parent-completed form, n=147		Telephone interview form, n=188		Direct interview form, n=116
	ESSENCE n=48	Non-ESSENCE n=99	ESSENCE n=88	Non-ESSENCE n=100	ESSENCE n=52	Non-ESSENCE n=64
Mean age in years (SD)	3.73 (1.18)*	2.67 (1.04)	4.04 (1.25)*	3.24 (1.34)	3.36 (1.18)	3.63 (1.33)
Male gender, n (%)	36 (75)**	52 (52.5)	60 (68.2)**	51 (51)	46 (71.9)**	25 (48.1)

Notes:

* P<0.05 for t-test pairwise comparisons in age between the ESSENCE and non-ESSENCE group;

** P<0.02 for chi-squared test in comparisons for gender between the ESSENCE and non-ESSENCE groups.

Abbreviation: ESSENCE, Early Symptomatic Syndromes Eliciting Neurodevelopmental Clinical Examinations.

Statistical analysis

The ESSENCE-Q total score was considered as a continuous variable. Cronbach’s α was used to test internal consistency. Confirmative factor analysis using Analysis of Moment Structures (AMOS) Version 7 was applied to test construct validity.²¹ A model represented by 12 items (ie, observable variables measuring specific aspects of child development) and 1 corresponding factor (ie, the overall development as a latent variable) was tested in AMOS. The model was separately tested for all 3 forms, and due to the non-normality present, the model was also evaluated using bootstrapping with 2,000 replicates.²² The results of both analyses yielded comparable conclusions and results reported here were obtained from the bootstrap analyses. The model fit to the data was evaluated using 4 fit indices: chi-squared/df ratio (<3 good), the comparative fit index (>0.90 acceptable, >0.95 excellent), Tucker–Lewis Index (>0.90 acceptable, >0.95 excellent), and root mean square error of approximation (<0.08 acceptable, <0.06 excellent).

Receiver operating characteristic curves were generated and the area under the curve (AUC) was compared to evaluate the validity of the 3 forms separately. Sensitivity and specificity values were calculated, with selected optimal cutoff values for screening purposes. Optimal cutoff values were selected based on 3 criteria: sensitivity and specificity should be above 0.8, sensitivity should be ≥ specificity, and Youden index ([sensitivity + specificity] − 1) should be as high as possible,^23,24 but preferably ≥0.70.

Results

Internal consistency and construct validity

Cronbach’s α was 0.91, 0.88, and 0.86 for the ESSENCE-Q parent-completed, telephone interview, and direct interview forms, respectively, with moderate-to-high item–total correlations (range 0.35–0.82).

The statistics assessing the adequacy of the model revealed good model fit for the ESSENCE-Q parent-completed and telephone interview forms, but poor model fit for the direct interview form (Table 3).

Table 3 Goodness of fit indexes for the ESSENCE-Q forms

ESSENCE-Q form	Chi-squared/df	CFI	TLI	RMSEA
Parent-completed*	1.95	0.95	0.93	0.08
Telephone interview**	2.19	0.92	0.90	0.08
Direct interview***	3.01	0.82	0.76	0.13

Notes: Root mean square error of approximation.

* Error correlations between items “general development” and “motor development/milestones”, items “activity (overactivity/passivity) or impulsivity” and “attention/concentration/listening”, and items “sensory reactions” and “mood”;

** Error correlations between items “social interaction/interest in other children” and “behavior”;

*** Error correlations between items “general development” and “motor development/milestones” and items “activity (overactivity/passivity) or impulsivity” and “attention/concentration/listening”.

Abbreviations: CFI, comparative fit index; ESSENCE-Q, Early Symptomatic Syndromes Eliciting Neurodevelopmental Clinical Examinations – Questionnaire; RMSEA, root mean square error of approximation; TLI, Tucker–Lewis index.

Sensitivity and specificity

The AUC of ESSENCE-Q parent-completed, telephone interview, and direct interview forms was (95% CI [CI]): 0.96 (0.93–0.99), 0.91 (0.86–0.95), and 0.91 (0.86–0.97), respectively.

Table 4 shows the most relevant cutoff values, sensitivities, specificities, and Youden indices for the ESSENCE scores of 2–6 points. Considering the predefined criteria, an optimal cutoff for the ESSENCE-Q parent-completed form of ≥3 had sensitivity of 0.96 (95% confidence interval [CI]: [0.86–0.99]), specificity of 0.84 (95% CI: [0.75–0.91]), and Youden index 0.80 (95% CI: [0.61–0.90]). The ESSENCE-Q telephone interview form had an optimal cutoff ≥4 based on the predefined criteria, but its diagnostic accuracy was greater with a cutoff ≥5 (Youden index 0.71), even its sensitivity (0.82, 95% CI: [0.72–0.89]) was slightly below the specificity (0.89, 95% CI: [0.81–0.94]). The ESSENCE-Q direct interview form with an optimal cutoff of ≥5 had sensitivity of 0.86 (95% CI: [0.77–0.96]), specificity of 0.88 (95% CI: [0.74–0.95]), and Youden index of 0.74 (95% CI: [0.52–0.85]).

Table 4 Cutoff, sensitivity, specificity, and Youden index values for the ESSENCE-Q parent-completed, telephone interview, and direct interview forms

Cutoff value	Parent-completed form			Telephone interview form			Direct interview form
	Sensitivity (95% CI)	Specificity (95% CI)	Youden index (95% CI)	Sensitivity (95% CI)	Specificity (95% CI)	Youden index (95% CI)	Sensitivity (95% CI)	Specificity (95% CI)	Youden index (95% CI)
≥2	0.98 (0.89–0.99)	0.78 (0.68–0.86)	0.76 (0.57–0.85)	0.93 (0.86–0.98)	0.67 (0.57–0.76)	0.60 (0.43–0.74)	0.94 (0.85–0.98)	0.46 (0.32–0.61)	0.40 (0.17–0.59)
≥3	0.96 (0.86–0.99)	0.84 (0.75–0.91)	0.80 (0.61–0.90)	0.87 (0.80–0.94)	0.73 (63.2–81.4)	0.62 (0.43–0.75)	0.92 (0.83–0.98)	0.65 (0.51–0.78)	0.57 (0.27–0.76)
≥4	0.88 (0.75–0.95)	0.92 (0.85–0.96)	0.80 (0.59–0.90)	0.84 (0.75–0.91)	0.82 (0.73–0.89)	0.66 (0.48–0.80)	0.89 (0.79–0.95)	0.81 (0.67–0.90)	0.70 (0.46–0.85)
≥5	0.83 (0.69–0.93)	0.94 (0.87–0.98)	0.77 (0.57–0.90)	0.82 (0.72–0.89)	0.89 (0.81–0.94)	0.71 (0.53–0.84)	0.86 (0.75–0.93)	0.88 (0.77–0.96)	0.74 (0.52–0.85)
≥6	0.77 (0.63–0.89)	0.95 (0.87–0.98)	0.72 (0.50–0.87)	0.78 (0.68–0.87)	0.90 (0.82–0.95)	0.68 (0.50–0.82)	0.78 (0.66–0.88)	0.90 (0.79–0.97)	0.68 (0.45–0.85)

Abbreviations: CI, confidence interval; ESSENCE-Q, Early Symptomatic Syndromes Eliciting Neurodevelopmental Clinical Examinations – Questionnaire.

Gender and age differences

The diagnostic accuracy for all 3 ESSENCE-Q forms was similar when screening boys and girls separately. All forms had slightly greater diagnostic accuracy for children aged 1–3 years than for those aged 3–6 years (Table 5). The significance of the differences was not tested due to small numbers.

Table 5 Sensitivity, specificity, and Youden index values for the ESSENCE-Q parent-completed, telephone interview, and direct interview forms considering gender and age groups

Group	Parent-completed form			Telephone interview form			Direct interview form
	Sensitivity (95% CI)	Specificity (95% CI)	Youden index (95% CI)	Sensitivity (95% CI)	Specificity (95% CI)	Youden index (95% CI)	Sensitivity (95% CI)	Specificity (95% CI)	Youden index (95% CI)
Boys	0.97 (0.86–0.99)	0.81 (0.68–0.90)	0.78 (0.54–0.89)	0.82 (0.69–0.91)	0.90 (0.79–0.97)	0.72 (0.48–0.88)	0.85 (0.71–0.94)	0.88 (0.68–0.95)	0.73 (0.53–0.85)
Girls	0.92 (0.62–0.99)	0.87 (0.74–0.95)	0.79 (0.36–0.94)	0.82 (0.63–0.94)	0.88 (0.75–0.95)	0.70 (0.38–0.89)	0.89 (0.66–0.99)	0.89 (0.74–0.98)	0.78 (0.52–0.89)
Age 1–3 years	0.94 (0.71–0.99)	0.92 (0.83–0.97)	0.86 (0.54–0.96)	0.87 (0.66–0.97)	0.96 (0.87–0.99)	0.83 (0.53–0.96)	0.86 (0.65–0.97)	100 (0.81–100)	0.86 (0.64–0.96)
Age 3–6 years	0.97 (0.83–0.99)	0.60 (0.39–0.79)	0.57 (0.22–0.78)	0.80 (0.68–0.89)	0.82 (0.68–0.91)	0.62 (0.36–0.80)	0.90 (0.77–0.97)	0.83 (0.66–0.93)	0.73 (0.57–0.86)

Abbreviations: CI, confidence interval; ESSENCE-Q, Early Symptomatic Syndromes Eliciting Neurodevelopmental Clinical Examinations – Questionnaire.

Discussion

The ESSENCE-Q is a brief screener for identification of children who may have neurodevelopmental difficulties. It was developed in this study for 7 South Slavic languages, namely, Bosnian, Bulgarian, Croatian, Macedonian, Montenegrin, Serbian, and Slovenian. This is the first time that 1 measure was simultaneously developed and validated for this language group and only the third psychometric study testing the ESSENCE-Q.

The internal consistency of the ESSENCE-Q was found to be good for all 3 forms, ie, direct or telephone interview or parent-completed, with moderate-to-high item–total correlations. In a study with the Japanese version, a parent-completed form, internal consistency was also good, although lower than in our study.¹² These data for internal consistency indicate that 12 ESSENCE-Q items have high homogeneity in measuring aspects of child development. In addition, we tested the construct validity of the ESSENCE-Q forms for the first time. The fit indices indicated acceptable fit of the data to the model for the parent-completed and telephone interview forms, but poor model fit for the direct interview form. In this regard, the latent underlying structure of neurodevelopment in the ESSENCE-Q is associated with 12, conceptually relevant items in the parent-completed and telephone interview forms, while in the direct interview form, the underlying structure may not be observed by all items. This result may indicate that the direct interview form could be conceptually different than the other 2 forms, with <12 items sufficient to represent neurodevelopment. In addition, there could be a genuine problem with the construct manifested through the direct interviewing, which includes some degree of combination of interview and observation, so that the actual recording of concern or not is, in fact, a partly investigator-based clinical judgment. Nevertheless, the observed difference could arise due to the sample size, which was smaller in the group of parents directly interviewed than for the other 2 groups.²⁵ Thus, additional factor analysis and item-response theory studies are needed to draw valid conclusions about the ESSENCE-Q and whether all forms have the same underlying construct of neurodevelopment. Further testing indicated that the accuracy of the ESSENCE-Q in differentiating children with NDDs and children with typical developmental trajectories was excellent for all 3 forms, but best for the parent-completed form (unlike in the Japanese study in public health, where the public health nurse had the “best” result).¹⁵ Analyzing the values for its sensitivity, specificity, and Youden index, the best cutoff values were found to be ≥3 points for the ESSENCE-Q parent-completed form and ≥5 points for the ESSENCE-Q telephone and direct interview forms. Nevertheless, the score of 2 points for the parent-completed form or 3 points for the interview form could also serve as good cutoffs, considering that its sensitivity and specificity values were above 0.87 and 0.65, respectively, which might be sufficient for screening NDDs. Hatakenaka et al¹² in Japan showed that optimal cutoff values could be 3–5 points for the parent-completed form used in a clinical setting. The same group demonstrated that for public screening, an optimal cutoff value could be that of 3 points for the parent-completed form or even 2 points for the interviews.¹⁵ We would recommend that a clinician or researcher uses higher cutoffs when screening in a clinical setting, because the percentage of false positives and false negatives would be lowest.

Finally, the results indicated that all 3 ESSENCE-Q forms could differentiate children with NDDs and children with typical developmental trajectories when gender was taken into account and its diagnostic accuracy was similar for boys and girls. In this regard, gender may not affect the validity of ESSENCE-Q screening, which is particularly relevant considering that some NDDs are more prevalent among boys than girls.¹ On the other hand, there was a difference when screening younger vs older children, and all 3 forms had slightly greater diagnostic accuracy for children aged 1–3 years than for those aged 3–6 years. Hatakenaka et al¹⁵ showed that there could be some cross-cultural differences and that, in Japan, the ESSENCE-Q might have better validity for screening boys and older children and cutoff values could be different when screening the same children at age 18 and 36 months of age. Accumulated evidence also indicates that the outcome of screening in NDDs could vary depending on children’s age and that diagnostic accuracy could be greater with more advanced age.^11,26 Thus, it would be relevant to take age into account when screening with the ESSENCE-Q.

Several limitations need to be considered. First, included children were not assessed through a unique diagnostic system or with a common diagnostic/psychological assessment throughout the included centers, whereas there was no such a common method to implement. This is particularly relevant for children with NDDs, whereas, it is possible to judge the presence and levels of the same NDDs based only on specific measurement methods. Second, convenience sampling in each country was considered and this is an important limitation also, because the assessment with the ESSENCE-Q could be affected by the diagnosis of or previous contacts with the research team of the child and the family included. In addition, an unequal age and gender distribution was present, which prevented us from creating distinct age × gender categories to test the measurement properties across these groups. Next, we did not simultaneously evaluate different forms for each child, for example a parent-completed and direct interview, in order to compare directly the measurement properties of the forms as it was done in the previous study.¹² Finally, the results could be biased due to the usage of the questionnaire’s data from similar languages, yet with different dialects.

Conclusion

This study provided additional data supporting good psychometric properties of the ESSENCE-Q as a screening questionnaire for NDDs. The ESSENCE-Q in the South Slavic languages developed here, namely, Bosnian, Bulgarian, Croatian, Macedonian, Montenegrin, Serbian, and Slovenian, is a promising brief screener for identification of children who may have neurodevelopmental difficulties in clinical settings, either as a parent-completed, telephone interview, or direct interview form. Recruiting more children and addressing the aforementioned limitations, we will continue with the project and further test the ESSENCE-Q in additional settings, such as primary care or public health screening.

Acknowledgments

We thank Daniela Todorova (pediatrician at Sahlgrenska University Hospital, Göteborg, Sweden), Nike Novak Vaca (psychiatrist at Sahlgrenska University Hospital, Göteborg, Sweden), Darko Sarovic (doctor and doctoral researcher at Gillberg Centrum, Göteborg University, Sweden), and Alen Salkic (psychologist at the Institute for Child and Adolescent Psychiatry, Skaraborg Hospital, Skövde, Sweden) for all the help they provided during the second phase of the pre-final questionnaire version harmonization from English, but also from the Swedish original version. Andrijana Dodik and Aneta Lakic (Clinic for Neurology and Psychiatry for Children and Youth, Belgrade, Serbia), Senka Naumovksa (NU NUB Sveti Kliment Ohridski, Macedonia), Staša Stropnik, Melita Bokalič, and Ana Mirković from Slovenia. Two abstracts of this paper were presented at the ESSENCE 2018 Conference on April 10–11, 2018, in Gothenburg, Sweden, as poster presentations with interim findings. The posters’ abstracts were published in the Program of the Conference and are available from: http://essence2018.se/wp-content/uploads/2018/04/ESSENCE2018-Program.pdf.

Disclosure

The authors report no conflicts of interest in this work.