Questionnaires are used both in research and in clinical practice as a way to gather information about possible mental health problems. For each question the informant is asked to select the most appropriate answer among the options presented. By giving a value to each of these options, responses can be condensed into scores. The interpretation of such scores depends on whether a given score (cut-off) or a combination of scores reflects specific problems. Often this is expressed as sensitivity and specificity in relation to a diagnosis or a set of diagnoses. Also, it may be informative to know how deviant a score is compared to the distribution of scores in the population. This is the rationale for national norms. However, establishing such norms is a difficult task where ideal standards have to be balanced against what is pragmatically possible. The challenges associated with obtaining good norm data are illustrated by a recent article in The Nordic Journal of Psychiatry focusing on The Strengths and Difficulties Questionnaire (SDQ) [Citation1].
SDQ is one of the most frequently used tools worldwide to screen for mental health problems among children and adolescents. It can be answered by parents, teachers and young people themselves. A complete set of Danish norms was published in 2019, based on more than 11,000 replies and stratified for age, sex, and respondents. These norms were based on a comprehensive dataset collected via public schools and day care facilities in a Danish municipality covering age-groups 2–17 and including children receiving special education [Citation2]. Until then only norms for two specific age groups had been available based on over 70,000 responses collected as part of large-scale cohorts [Citation3].
The referred article compares these two sets of norms and demonstrates how the cohort-based norms more often than the municipality-based norms characterize a given score as deviant [Citation1]. The authors also present data collected as part of the large Danish National Birth Cohort study. It is stressed how the data more resembles the earlier cohort-based norms than the later municipality-based norms. They recommend that a revised set of norms should be based on the large datasets available from Danish cohort studies. As valuable as large cohort studies may be, they are, however, far from ideal as basis for norms. The datasets only relate to some age groups and do not include all informants (parents, teachers, self-reports). Even more important, the recruitment into the studies and the need for recurrent data collection pose significant challenges when it comes to representativeness. The data from the Danish National Birth Cohort study presented in the article may serve as an illustration. When this study was initiated, it was not expected that the cohort would provide a representative sample of the population. The intention was to use the data for internal comparison. Subsequently, it was also found that in addition to a low overall participation rate (31%) for entering the study, women with few resources were strongly underrepresented [Citation4]. Further attrition was seen when mothers were asked to fill out questionnaires several years later. When it comes to norms, the importance of high response rates was demonstrated by the results from a Norwegian population study obtaining an almost complete set of SDQ responses from teachers. Significantly more difficulties were reported for children where parental responses were missing compared to where they were present [Citation5].
Referring to limitations in the data available, the authors also suggest that norms should be based on data relating to singletons and exclude children with various difficulties. This approach directly contradicts the understanding of norms as a description of the distribution of scores in the population in its entirety.
Finally, the proposed norms would be based on data collected within a 20-year time interval. This would disregard the fact that behaviour, the school context (including a new school reform applied in Denmark in 2012), as well as parents’ and teachers’ rating of children, change over time.
To sum up, norms will always constitute a non-perfect estimation of the current distribution of scores in the target population and may therefore be improved. However, even very large numbers of participants cannot compensate for low (selective) response rates and exclusion of individuals with a known high risk of mental health problems.
Disclosure statement
No potential conflict of interest was reported by the author(s).
References
- Scheel Rasmussen I, Strandberg-Larsen K, Overbeck G, et al. A critical examination of Danish norms for the strengths and difficulties questionnaire (SDQ). Nord J Psychiatry. 2023;77(8):818–823. doi: 10.1080/08039488.2023.2250316.
- Arnfred J, Svendsen K, Rask C, et al. Danish norms for the strengths and difficulties questionnaire. Dan Med J. 2019;66(6):A5546.
- Niclasen J, Teasdale TW, Andersen A-MN, et al. Psychometric properties of the Danish strength and difficulties questionnaire: the SDQ assessed for more than 70,000 raters in four different cohorts. Plos ONE. 2012;7(2)
- Jacobsen TN, Nohr EA, Frydenberg M. Selection by socioeconomic factors into the Danish national birth cohort. Eur J Epidemiol. 2010;25(5):349–355. doi: 10.1007/s10654-010-9448-2.
- Heiervang E, Stormark KM, Lundervold AJ, et al. Psychiatric disorders in Norwegian 8- to 10-year-olds: an epidemiological survey of prevalence, risk factors, and service use. J Am Acad Child Adolesc Psychiatry. 2007;46(4):438–447. doi: 10.1097/chi.0b013e31803062bf.