1,116
Views
0
CrossRef citations to date
0
Altmetric
Research Articles

Assessment fidelity of a language screening instrument for 4-year-olds

, &
Pages 189-196 | Received 18 Jun 2021, Accepted 20 May 2022, Published online: 01 Jun 2022

Abstract

Aim

The aim of the study was to explore the assessment fidelity of Språkfyran, a language screening instrument for four-year-old children. Språkfyran is a mandatory part of the healthcare program within the Swedish Child Health Service (CHS) and is offered to all four-year-olds in the region Scania in Sweden.

Methods

The study was based on structured observations of twenty-four specialist CHS nurses’ adherence to the Språkfyran protocol during screening.

Results

All the observed nurses deviated from the test protocol. There was a large variation in the number of deviations from the test protocol per nurse, with the highest number of deviations occurring for three specific testing items. Significantly more deviations were made with four-year-old bilingual children as opposed to four-year-old monolingual children. Half of the nurses did not use the test protocol.

Conclusions

There is a clear need to improve the assessment fidelity of Språkfyran. Both the training that the nurses are offered, and the development of the test, are essential in securing the aim of high-quality work within the CHS. Support from experts in child speech-language development and disorders is suggested to be available at the CHS in Sweden.

Background

Speech and language disorders are common neurodevelopmental disorders [Citation1]. The prevalence of speech disorders in four to five-year-olds has been estimated to be about 2–3% [Citation2,Citation3]. Speech disorders often co-occur with language disorders (LD), especially in males [Citation2]. The prevalence of LD varies with the definition being used. In a British population-based survey, which included over 7,000 children aged between 4:9 and 5:10 years, the authors reported a 2.3% prevalence of LD associated with intellectual disability and/or other medical developmental conditions, and 7.6% prevalence of developmental language disorder (DLD) of unknown origin [Citation1]. These British prevalence rates for LD and DLD are in line with previous research from the US [Citation4] and from Germany [Citation3]. For some children, deficits in speech, language and communication functions are part of a neurodevelopmental disorder, such as autism spectrum disorder, or intellectual disability. For other children, DLD exists as a primary challenge in the child’s development, and in the absence of other clinical explanations including intellectual disability or acquired brain conditions [Citation5].

In Sweden, the United Nations Convention of the Rights of the Child guides the work of the Child Health Service (CHS) [Citation6]. To accomplish the aims of all children’s equal worth and rights, the Swedish CHS offers a program, free of charge, to all children in Sweden, from newborns until about six years of age [Citation6]. Following a national program, screening of speech and language development is offered to children at several particular ages at local child health centres (CHC). Most children in Sweden (>99%) [Citation7] attend the CHS programme [Citation6]. With some minor regional differences regarding the ages when language screenings occur, language development is followed from infancy until five years of age [Citation8]. Specialist nurses are in charge of most of the health appointments at the CHC, including administration of the language screenings. The purpose of the language screenings is to identify those children who should be referred to a specialist for more detailed assessment [Citation9].

The present study focuses on the language screening offered to all four-year-old children during a broader health appointment at their local CHC in the region Scania. Scania is the southernmost region of Sweden, with approximately 160 CHC. During the last ten years, about 15 500 children have been born per year in Scania, corresponding to about 13% of the children born per year in Sweden [Citation10]. Until 2015, there was no validated or mandated language screening method for four-year-olds used by the CHS in Sweden [Citation8]. After a regional decision in Scania in January 2016, a validated language screening instrument, Språkfyran (In English: LanguageFour) [Citation11], has been a mandatory part of the healthcare program. All CHS nurses in Scania are offered a workshop where they learn how to administer Språkfyran.

Good validity and high reliability of a screening instrument is required for correct identification of poor speech and/or language development in children. In addition, nurses assessing children’s language skills need to know how to administer the testing items of the specific instrument they are using, and they must also know how to analyse and act on the test results. High assessment fidelity is an important aspect of validity and thus a quality assurance of a screening procedure essential for the public health programs [Citation6]. Assessment fidelity can involve, for example, the administrators’ adherence to the instructions of the test items, adherence to the scoring, and to the procedure of how to act on the test results. Other relevant information to report on assessment fidelity can include the qualification of the test administrators and the quality of the training to administer the tool. None of these aspects of the assessment fidelity have yet been reported for Språkfyran. The main aim of the current study is to observe nurses’ adherence to the Språkfyran protocol regarding their administration of the instructions of each test item.

Nayeb et al. [Citation12] reported that CHS nurses systematically simplified items in language screening tests when assessing bilingual children as compared to monolingual Swedish-speaking children, and that the nurses were less inclined to refer bilingual children to a speech-language pathologist as compared to monolingual Swedish-speaking children. The explanation was found to be the nurses’ lack of knowledge of children’s bilingual language development as well as a lack of training in test administration [Citation12]. This shows that the complexity of different factors involved in a language screening procedure may affect the outcome and consequences of the screening routine within CHS. A study by Wiefferink et al. [Citation13] reported caseload characteristics of over 11,000 children (2–7 years) referred to speech-language pathology services in Holland and found that bilingual children were referred later than monolingual children, and the bilinguals who were referred to speech-language pathology services tended to have more severe LD than the monolinguals. The results of the study indicated a need to increase the CHS’s awareness of childhood bilingual development. Moreover, the authors suggested implementation of a language screening instrument designed for health professionals with no expertise in speech-language pathology, in addition to receiving training in identifying bilingual children with less severe language disorders. The findings in the Dutch study [Citation13] are corroborated by a similar Swedish study by Salameh and colleagues [Citation14] which reported lower odds for preschool-age bilingual children to be referred by CHS to a speech-language pathologist, as compared to monolinguals. Also, bilinguals were older and more often diagnosed with severe problems than monolingual children. Taken together, previous research indicates a risk of inequity in the service provided from CHS to bilingual children as compared to monolingual children. The second aim of the current study is to compare the fidelity of CHS nurses’ adherence to test instructions during the language screening of monolingual versus bilingual children.

Aims

The aim of the present study was to explore the CHS nurses’ adherence to the protocol of the Språkfyran, when administering the screening to four-year-olds, and answered the following questions: (1) Do the CHS nurses follow the Språkfyran’s protocol when administering the instruction for each test item, and if not, in what way and for which items do they not adhere to the protocol? (2) Is there any relationship between the CHS nurses’ adherence to the test protocol if the child is monolingual Swedish-speaking or bilingual with Swedish as one of the child’s languages?

Method

Design and recruitment

Structured observations were used to study CHS nurses’ adherence to the Språkfyran protocol during screening. The observations were conducted during three months (August–September–October, 2019). Twenty-three of the largest health centres in one of Scania’s four districts were invited to participate via e-mails to the heads. The heads of 14 of the 23 health centres gave their consent allowing us to recruit participants, and 37 CHS nurses were available to be contacted. Of these, 28 nurses (76%) working at 11 health centres, gave their written and oral informed consent to participate in the study. Four of these 28 CHS nurses did not have any four-year-old child scheduled during the study time, which resulted in inclusion of 24 CHS nurses, who could be observed during one screening session each. All participating nurses fulfilled the National Swedish Board of Social Health and Welfare’s competence demands [Citation6] which are required for working as a CHS nurse in Sweden. Additionally, CHS nurses specialize in either pediatric nursing or district nursing. Immediately before the observation, the observer (the first author who was a Master’s student and experienced clinical nurse) asked each nurse about their length of work experience as a CHS nurse, and if the nurse had participated in a workshop on how to administer Språkfyran, or not. The nurse was also asked by the observer if the child who was about to attend the screening was mono- or bilingual. The children were categorized as either monolingual Swedish-speaking, or bilingual when speaking or being regularly exposed to two or more languages. At the time of the child’s visit to the CHC, the observer gave both oral and written information to the parents, and oral information to the child about the study, and asked for their oral consent. The bilingual children’s accompanying parents could all communicate and give their consent in Swedish. The information to the parents and the child emphasized that the focus of the observation was on the CHS nurse, and that no data would be collected that would reveal either the child’s or the parents’ identities. The study followed the declaration of Helsinki [Citation15] and was approved by the local Advisory Committee on Research Ethics in Health Education.

Measures

Språkfyran is developed solely for CHS health appointments for four-year-old children [Citation16]. It consists of 32 items and includes the following categories: Word and sentence comprehension, lexical organization, repetition of words and nonwords, and identification of a central theme in a picture, and lexical access. Språkfyran takes about 5–15 min to administer including an additional five final items. These five items require the nurse to judge the child’s intelligibility, prosody, and their ability to cooperate and handle turn-taking during the assessment. The scoring is binary, with 0 or 1 points per item: resulting in a maximum score of 37. The cut-off point for typical speech-language development is 33 out of 37 points. A score below 33 indicates risk of poor speech-language skills and advises that the child should be referred to a speech-language pathology clinic by the CHS nurse.

The test norms are based on 342 children aged 3:10 to 4:4 years, including 77% monolingual and 23% bilingual children. The average total scores from mono- and bilinguals did not differ significantly, despite that not all bilinguals were born in Sweden and/or had attended a Swedish preschool [Citation11]. In a study of the clinical accuracy of Språkfyran, based on 328 participants, the test developers reported a sensitivity of 86% and a specificity of 96%, and a Cronbach’s Alpha of .80 [Citation11]. The interrater reliability was high: Based on 37 cases scored independently by two nurses the intraclass correlation coefficient (ICC) was .99 (p < .001) [Citation11].

In the test manual [Citation16] the test administrator is instructed to request the accompanying caregiver to take a passive role during the assessment, meaning that neither the caregiver nor the test administrator (i.e. the CHS nurse), should help or comment on the child’s responses during the screening procedure. The test should be administered at a calm rate, and any response from the child should be accepted regardless of whether it is correct or not. Moreover, according to the test manual the nurses should be aware that it is natural to want to help a child who does not manage a task, but such help should be avoided.

Observation protocol

An observation protocol was developed by the first author based on the Språkfyran protocol form that the CHS nurses were to fill in during the screening. The observation protocol was checked for fidelity to the Språkfyran protocol by the last author. A pilot test of the protocol was performed during two observations. From the results it was concluded that there was no need for any revisions of the protocol; thus, the two observations were included in the study. During the screening, the observer noted for each item in the study protocol whether the CHS nurse followed the test protocol or not, the child's response, and how the nurse responded to the child’s behaviour, e.g. with verbal encouragement, or explanations of the instructions. The child’s gender (male/female), age (year:months), and if the child was referred to a speech-language pathologist after the screening (yes/no) was noted for each child by the observer. Finally, the total score of Språkfyran, according to the observer, was documented. The observations of the screening lasted for an average of 15 min.

To minimize the risk of intruding on the clinical situation, the observer was seated with as much distance as possible (depending on what the room permitted) parallel to the CHS nurse and the child. This triangle positioning was aimed to assure that the communication between the child and the CHS nurse was in focus for the child’s attention. The observer was silent and careful not to seek any contact with the individuals in the room during the whole visit.

Statistical analyses

The data analyses were performed in the Statistical Package for the Social Sciences (SPSS, version 27). The data was analysed using descriptive statistics. Pearson Correlation was used to measure the association between the number of years the nurse had worked within the CHS with the number of deviations made per nurse. The Mann-Whitney U-test was used for the comparison of the number of nurses’ deviations between groups of children based on whether the children were mono- or bilingual, and for the comparison of mono- versus bilingual groups of children’s responses on a specific task. Non-parametric tests were used for data analysis because the data were not equally distributed. The p-value of <.05 was viewed as statistically significant.

Results

Background data

A total of 24 observations of screenings with Språkfyran administered by 24 CHS nurses were conducted and included in the analysis. The nurses’ experience of working within the CHS ranged from six months to 21 years (mean 7.0, SD 5.1). Twenty-one of the nurses (87.5%) had participated in a workshop on how to administer Språkfyran. The children’s ages ranged from 3:11 years to 4:6 years (mean 4.0, SD 0.14). All but one child, who was 4:6 years, were within the standardized age reference for Språkfyran, i.e. 3:10 to 4:4 years. The gender ratio was 46% males (n = 11) and 54% females (n = 13). The majority were bilingual (58.3%, n = 14).

Nurses’ adherence to the test manual

All nurses made deviations from the test manual, though there was a large variation in the number of deviations per nurse during the observed session. The deviations ranged from two to 29 (mean 10.1, SD 7.2), with up to six deviations per test item (range 0–6).

The results of the Pearson test showed no association between the number of years the nurse had worked as a CHS nurse with the number of deviations they made (r=.206, p=.335). The three nurses who had not yet taken part in a workshop on how to administer Språkfyran were among the nurses with the lowest number of deviations: These three nurses made two, two, and eight deviations, respectively.

Only half of the nurses used the test protocol to fill in the child’s responses while administering the screening. The other 12 nurses neither used the protocol, nor made any written notes of the child’s responses. Of the 12 nurses who used the test protocol during screening, 42% (n = 5), also scored the last five items during the screening. This means that the form was not filled in for the last five items by 19 out of 24 nurses (79%).

Deviations per test item 1 to 32

shows the percentage of nurses deviating at least once per test item. The test items in which most of the nurses deviated from the manual were items eight (n = 21, 88% of the nurses) and nine (n = 18, 75%). These two items aim to assess the child’s ability to categorize nouns [Citation16]. For example, in item eight, four cards with pictures of clothes and four with animals are laid on the table in front of the child. Next, the child’s attention is drawn to two other pictures, one of a boy and one of a house. The child is instructed to give the clothes to the boy and make all animals go to their house. A third item which resulted in a high number of deviations was item 10 (n = 15, 63%) which is the first of four items asking the child to pick the odd one out of four pictures. Of all test items, only one (no. 29) was administered without deviations by any of the nurses. Item 29 is the ninth of 10 items in a row assessing the child’s ability to repeat nonwords.

Figure 1. Percentage of 24 nurses deviating at least once from the test manual from test item 1 to 32.

Note. Black bars (item 1–7) word and sentence comprehension. White bars (item 8–14) lexical organization. Grey bars (item 15–20) word repetition. Striped bars (item 21–30) nonword repetition. Dotted bars (item 31–32) thematic picture.

Figure 1. Percentage of 24 nurses deviating at least once from the test manual from test item 1 to 32.Note. Black bars (item 1–7) word and sentence comprehension. White bars (item 8–14) lexical organization. Grey bars (item 15–20) word repetition. Striped bars (item 21–30) nonword repetition. Dotted bars (item 31–32) thematic picture.

The type of deviations

The observations of the nurses administering the screening resulted in three different categories of deviations from the manual: (I) Repetition of the instruction, (II) Clarification or simplification of the instruction, and (III) Positive encouragement to influence the child to try to respond differently. A typical example includes all three categories of deviations while administering one single item: The nurse did not accept the child’s answer, but instead made further explanations of the instructions and thereafter repeated the new instruction. For example, when it came to one of the items with the highest number of deviations, item 10, the nurses tended to deviate from the manual by clarifying the question, changing the instruction to, “which three pictures fit together,” and repeatedly encouraged the child to respond differently.

Is there any relationship between the CHS nurses’ adherence to the test manual if the child is Mono- or bilingual?

We found a higher number and a larger variation of the number of deviations made by the nurses when assessing bilingual children (mean 12.2, range 2–29), as compared to when assessing monolingual children (mean 5.6, range 2–9). This means that deviations were 2.2 times as common when the nurses assessed bilingual children as compared to monolingual children. A Mann-Whitney U-test indicated the difference was statistically significant (U = 31.5, Z = –2.262, p = .022). In addition, the observations also revealed that the nurses asked the parents of bilingual children to translate the instruction to the child’s first language in cases where the child could not understand an item in Swedish.

Other observations

The observer scored the children’s responses to the items one to 32 (). The item with the lowest number of correct responses was item 10. This same item also was the third most common to exhibit nurses’ deviations. Only 58.3% of the children gave a correct answer to item 10. Moreover, there was a higher proportion of monolingual children (n = 8, 80%) than bilingual children (n = 6, 43%) managing the task, though Mann-Whitney U-test indicated the difference between groups was not statistically significant (U = 44.0, Z = –1.781, p = .138). Three children (12.5%) were referred to speech-language pathology services.

Figure 2. Percentage of 24 children, who responded correctly to each of the 32 items.

Note. Black bars (item 1–7) word and sentence comprehension. White bars (item 8–14) lexical organization. Grey bars (item 15–20) word repetition. Striped bars (item 21–30) nonword repetition. Dotted bars (item 31–32) thematic picture.

Figure 2. Percentage of 24 children, who responded correctly to each of the 32 items.Note. Black bars (item 1–7) word and sentence comprehension. White bars (item 8–14) lexical organization. Grey bars (item 15–20) word repetition. Striped bars (item 21–30) nonword repetition. Dotted bars (item 31–32) thematic picture.

Discussion

This study investigated the assessment fidelity of the language screening test Språkfyran, based on structured observations of 24 nurses administering the screening to four-year-olds. All nurses made deviations from the test protocol but there was a large variation in the number of deviations across all nurses. Although all but three of the 24 nurses had participated in a workshop on how to administer Språkfyran, every nurse made deviations from the test manual and there was no significant association with the nurses’ number of years working within CHS. It could be expected that with more years of experience the nurses would be more aware of the importance of following test protocols for their different screenings. It was even so that two of the three nurses who had not yet attended the workshop deviated the fewest number of times of all nurses (“only” twice each), while the third of the nurses deviated 8 times. Of all 24 participants, these three nurses had, in fact, worked for the shortest amount of time (6 months to 2 years) within the CHS.

Reasons for health professionals not following guidelines have been reported, e.g. as a result of a lack of awareness of the content or procedures of the guideline, lack of outcome expectancy, lack of agreement with the value of the proposed guideline and the inertia of old practice habits [Citation17]. To counteract such behaviour, it would be valuable to improve the training by adding an examination component to the workshop where each participating nurse is observed while administering Språkfyran and is given feedback. The workshop would then be followed up with both self-assessment and performance assessment with feedback for further self-directed learning [Citation17,Citation18], as well as booster training sessions for scoring and administration [Citation19]. This would be a way to underline how important it is to find those children, both mono- and multilingual, who need to be referred, on quality-assured grounds, to a speech-language pathologist.

Of the participating nurses only 50% used the screening protocol for scoring and made notes during the screening. In addition, the results also showed that only six out of 24 nurses used the protocol for the overall assessment of the child’s intelligibility, prosody, ability to cooperate, and ability to handle turn-taking. This indicates they ignored the test items 33–37 during the screening procedure. The documentation was made in each child’s medical journal after the end of the visit. It can then be assumed that the nurses made an overall assessment of the final five items based on the whole visit, and not on the screening. This deviation from the protocol, in addition to not using the protocol to make notes during the test, could negatively affect the reliability of the screening instrument [Citation18]. The results from the screening are therefore not reliable as a basis for referral to a speech-language pathologist. The findings in the present study strongly indicate that there is a need to improve the assessment fidelity of Språkfyran.

Any screening tool used by the public health system should be simple to administer and easy to understand by both the patients and health professionals [Citation20,Citation21]. Considering that Språkfyran is offered to all four-year-old children in Scania each year the nurses’ experiences of language screenings [Citation22] should be considered in further development of this tool. Some of the deviations may be relevant and thereby possibly important in that they may inform the test developers on how to improve the test manual, items, or the formal training on how to administer the test. Moreover, since Swedish CHS nurses are not experts in speech-language development and disorders, support should be available for the nurses. This could be provided by, for example, clinical speech-language pathologists [Citation13] who specialize in mono- and bilingual child language development and disorders. However, with very few exceptions, the CHC in Sweden don’t have speech-language pathologists among their staff.

In a study of the clinical accuracy of Språkfyran, authored by the developers, two major limitations of the test were discussed [Citation11]. Firstly, there was a lack of cross-professional, independent inter-rater reliability testing of the scores. Secondly, there was a lack of blinded follow-up assessments of the participants who had a poor result on the screening, and no follow-up assessment of children with adequate results were conducted. But cross-professional inter-rater reliability based on nurses and speech-language pathologists seems inappropriate since Språkfyran is developed for CHS nurses, and not for speech-language pathologists. There is a need for language screening instruments specifically designed for non-speech-language pathologists [Citation13]. The psychometrics of a screening instrument are important to secure, especially when introduced as a mandatory tool [Citation23] involving all four-year-olds [Citation7], i.e. approximately 17,000 four-year-old children per year in Scania [Citation10]. Inter-rater reliability checks during the development of a new tool are not enough to ensure assessment fidelity, and the consequences of low fidelity of a test comprises risk of, e.g. unearned clinical accuracy [Citation24].

While administering the test to bilingual children, the nurses made significantly more deviations from the test protocol compared to the monolingual children. These results correspond with a previous Swedish study showing that 82% CHS nurses (n = 863) simplified screening procedures while performing language screenings with bilingual children, even for bilingual children with adequate Swedish language skills [Citation12]. Nayeb et al. [Citation12] reported that the strongest predictor of the nurses’ simplified language screening practices was the nurses’ belief of language development being slower in bilingual children. This misconception of bilingual language development [Citation25] could result from inadequate, or insufficient continuous professional training [Citation13]. Irrespective of the reasons for the higher number of deviations from the protocol when assessing bilingual children, the findings in the present study indicate a risk of inequity in the language screening program at the CHC in Scania. This implies that it might not be enough to learn how to administer a language screening as a guarantee of an adequate assessment, since the nurses’ attitudes of children’s bilingual competence also seems to be of decisive importance. A risk of systematic discrimination of bilingual children within Swedish CHS could be avoided, for example, by raising the awareness of bilingual language development. This aspect of quality assurance must also be scrutinized in workshops for test administration of language screenings, such as Språkfyran. Most importantly, the nurses should be informed of which pitfalls to avoid.

There are well-known challenges in language assessments of bilingual children [Citation26,Citation27], for example that norm-referenced assessments can be inadequate to use, e.g. because of linguistic bias and problems with the representations of bilingual children in normative samples [Citation28]. The 23% comprised of bilinguals in the normative sample of Språkfyran, representing 29 different languages, are described as “representative of the Swedish population” [Citation11]. Perhaps, the representativeness refers to the proportion of bilinguals. However, since there are no official statistics on the number of bilingual preschool-age children in Sweden, the current best approximation is the statistics of children with a foreign background. During 2016 to 2020 about 30–32% of all four-year-old children in Scania had a foreign background, meaning that they were either born abroad, or they had at least one parent born abroad. Corresponding statistics regarding all four-year-olds in Sweden was 23–27% during the same years [Citation29]. Obviously, not all children with a foreign background are bilingual; neither are all children with both parents born in Sweden monolingual Swedish-speaking. The point is, bilingual children in Sweden should not be referred to as a homogenous group, because they differ on the types as well as status of their languages, and on the quality and quantity of the exposure of their languages [Citation25].

Moreover, identification of child language disorders can be challenging also due to social factors [Citation30,Citation31]. For example, when distinguishing between a language disorder and poor performance in a second language, the context of a bilingual child’s language learning needs to be considered, and assessment solely on the second language is not optimal. According to the authors of Språkfyran, “Screening in a second language is not ideal” [Citation11]. The same authors suggest dynamic assessment as an alternative method [Citation11]. However, dynamic assessment of language skills is very time consuming, demands expertise, has low inter-rater reliability, and is mainly used for diagnostic therapy [Citation26]. Thus, dynamic assessment cannot be an alternative in public health screening. The Swedish national guidelines for CHS nurses on clinical practice for language screening of bilingual children [Citation32] recommend a holistic approach in the language screening of bilingual children. This is in line with a recent review of assessment approaches to use with bilingual children by speech-language pathologists [Citation26].

The fact that nurses asked some of the bilingual children’s parents to translate the instructions to the child’s first language, potentially indicates that the nurses are aware that the test does not assess language development in bilingual children, but competence in Swedish. If this is true, then the observation has implications for the construct validity of Språkfyran when administered to bilingual populations. Asking the parents to translate the instructions of Språkfyran is problematic in terms of appropriateness: Being developed for Swedish, it is not suitable for translations to other languages. In public screening, false negatives are often considered a larger concern than false positives, as their deficits risk to go unrecognized [Citation9]. However, if a screening tool assesses children’s second language competence, and not their language development, then there is an impending risk of a higher number of false positives among minority populations for whom the language of the screening tool is not their first language [Citation33].

The three items for which most nurses deviated were all within the category of the six items aiming at assessing two aspects of lexical organisation. Also, for the remaining three items within this category a relatively high number of nurses deviated. This means that the lexical items seem to be the least reliable of all items, and the finding is pointing to potential problems in terms of construct validity for this category of items, especially for bilingual children. Despite the repetitions, clarifications, and simplifications of the instructions, these items were also among the most difficult for the children to manage.

Furthermore, the first of the four lexical items where the child is instructed to pick the odd one out showed to be significantly more difficult to fulfil than the three following items in the same category. This is indicating that the first item within these tasks function as an exercise item for the children to manage the following three. This implies that there is a need for an exercise task preceding the items 10 to 13.

Limitations

This observational study included only 24 nurses, and although they were representative for CHC locations in both rural and urban areas in one of the four Scanian districts, they neither represented all nurses in the district, nor all other CHC locations in Scania. Moreover, the aim of the study was transparent for the participants, and this may have affected the validity of the study. If there was such a bias, the nurses would have rather tried to hold back the deviations, as opposed to continuing to enact them. However, we do not know if the nurses were aware that they deviated from the manual, or if they were aware of the number of deviations they made during the observations.

This study focuses on the procedure of the screening but lacks information regarding the nurses’ scoring per item. In a future study, this information should be included because it could clarify the relationship between the two, i.e. how assessment fidelity such as adherence to the protocol, affects the scoring. Since neither the nurses’ scoring sheets, nor information about the basis for their decisions for the referrals of three children to speech-language pathology services was collected, the referral rate cannot be analysed based on assessment fidelity.

In future studies, the observation protocol could include detailed aspects of the deviations, such as the type of encouragement or the simplification of an item. This information could be valuable for those who organize future workshops for nurses on how to administer Språkfyran. Moreover, the types of deviations in relation to specific items and item categories could be valuable to investigate and can provide advice for further development of this screening instrument.

Conclusions

Speech and language disorders are relatively common [Citation1] and therefore constitute a significant concern in public health [Citation23]. Poor speech, language and communication skills are associated with a risk of behavioural and social difficulties, and may be early markers of neurodevelopmental disorders, such as autism spectrum disorder or intellectual disability. Suggested improvements, both regarding the training nurses are offered and the development of the test, are essential in securing the aim of high-quality work within the Swedish CHS. The professional standard of Swedish CHS nurses requires a non-discriminatory service and competence in carrying out the health programs [Citation6]. Further research is required to study what these improvements and further development of the test implies.

Acknowledgement

We are grateful to the study participants, and the families for their cooperation. Thank you to speech-language pathologist Allyson Plumberg, M.S., CCC-SLP, for her help with language revision of the manuscript.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Notes on contributors

Johanna Dahlén

Johanna Dahlén, MSc, Public Health Nurse, Samarithemmets Child Health Care Center, Region Uppsala, Sweden.

Eva Drevenhorn

Eva Drevenhorn, PhD, Public Health Nurse, Associate Professor at the Department of Health Sciences, Faculty of Medicine, Lund University, Sweden.

Nelli Kalnak

Nelli Kalnak, PhD, Speech-Language Pathologist, Clinical Specialist in Neurodevelopmental Disorders. Researcher at the Department of Women's and Children's Health, Karolinska Institutet, and Helsingborg Hospital. Research focus on developmental language and reading disorders.

References

  • Norbury CF, Gooch D, Wray C, et al. The impact of nonverbal ability on prevalence and clinical presentation of language disorder: evidence from a population study. J Child Psychol Psychiatry. 2016;57(11):1247–1257.
  • Eadie P, Morgan A, Ukoumunne OC, et al. Speech sound disorder at 4 years: prevalence, comorbidities, and predictors in a community cohort of children. Dev Med Child Neurol. 2015;57(6):578–584.
  • Weindrich D, Jennen-Steinmetz C, Laucht M, et al. Epidemiology and prognosis of specific disorders of language and scholastic skills. Eur Child Adolesc Psychiatry. 2000;9(3):186–194.
  • Tomblin JB, Records NL, Buckwalter P, et al. Prevalence of specific language impairment in kindergarten children. J Speech Lang Hear Res. 1997;40(6):1245–1260.
  • Bishop DV. What causes specific language impairment in children? Curr Dir Psychol Sci. 2006;15(5):217–221.
  • National Board of Health and Welfare. Vägledning för barnhälsovården. [guidelines for the child health service.]. Sweden: NBHW; 2014.
  • Svenska barnhälsovårdsregistret [Swedish child health service register]. Årsrapport svenska barnhälsovårdsregistret 2019 [annual report from the Swedish child health service register 2019]. Sweden. 2019.
  • Fäldt A. Rikshandboken för barnhälsovård: Språkundersökning och screening [National handbook for child health service: Language examination and screening] [Internet]2019. May 14 [cited 2021 Nov 1]. Available from: https://www.rikshandboken-bhv.se/metoder–riktlinjer/sprakundersokning-och-screening/. Swedish.
  • Neisworth JT, Bagnato SJ. The mismeasure of young children: the authentic assessment alternative. Infants Young Child. 2004;17(3):198–212.
  • Statistics Sweden [Internet]. Population Statistic for 2010–2020 [cited 2021 Nov 1]. Available from: http://www.scb.se.
  • Lavesson A, Lövdén M, Hansson K. Development of a language screening instrument for swedish 4-year-olds. Int J Lang Commun Disord. 2018;53(3):605–614.
  • Nayeb L, Wallby T, Westerlund M, et al. Child healthcare nurses believe that bilingual children show slower language development, simplify screening procedures and delay referrals. Acta Paediatr. 2015;104(2):198–205.
  • Wiefferink K, van Beugen C, Wegener Sleeswijk B, et al. Children with language delay referred to dutch speech and hearing centres: caseload characteristics. Int J Lang Commun Disord. 2020;55(4):573–582.
  • Salameh EK, Nettelbladt U, Håkansson G, et al. Language impairment in Swedish bilingual children: a comparison between bilingual and monolingual children in Malmö. Acta Paediatr. 2002;91(2):229–234.
  • World Medical Association. World medical association declaration of Helsinki: ethical principles for medical research involving human subjects. JAMA. 2013;310:2191.
  • Lavesson A, Hansson K, Lövdén M. Språkfyran [LanguageFour]. Lund: Kunskapsutveckling i Lund AB; 2015.
  • Davis DA. How to help professionals maintain and improve their knowledge and skills: triangulating best practices in medicine. In Ericsson KA, editor. Development of professional expertise: toward measurement of expert performance and design of optimal learning environments. Cambridge: Cambridge University Press; 2009. p. 180–202.
  • Stone-MacDonald A, Pizzo L, Feldman N. Fidelity of implementation in assessment of infants and toddlers: evaluating developmental milestones and outcomes. Cham: Springer; 2018.
  • Richardson JD, Hudspeth Dalton SG, Shafer J, et al. Assessment fidelity in aphasia research. Am J Speech Lang Pathol. 2016;25(4S):S788–S797.
  • Mattsson CM, Mårild S, Pehrsson NG. Evaluation of a language‐screening programme for 2.5‐year‐olds at child health centres in Sweden. Acta Paediatr. 2001;90(3):339–344.
  • Wilson JMG, Jungner G. 1968;Principles and practice of screening for disease. Geneva: World Health Organization; (Public health papers, 05556015;34)
  • Fäldt A, Nordlund H, Holmqvist U, et al. Nurses’ experiences of screening for communication difficulties at 18 months of age. Acta Paediatr. 2019;108(4):662–669.
  • Fäldt A. Targeting toddlers’ communication difficulties at the swedish child health services–a public health perspective [dissertation]. Uppsala: Acta Universitatis Upsaliensis, Sweden; 2020.
  • Shadish WR, Cook TD, Campbell DT. Experimental and quasi-experimental designs for generalized causal inference. Boston (MA): Houghton Mifflin; 2002.
  • Kohnert K, Ebert KD, Pham GT. Language disorders in bilingual children and adults. 3rd ed. San Diego (CA): Plural Publishing, Inc; 2021.
  • De Lamo White C, Jin L. Evaluation of speech and language assessment approaches with bilingual children. Int J Lang Commun Disord. 2011;46(6):613–627.
  • Caesar LG, Kohler PD. The state of school-based bilingual assessment: actual practice versus recommended guidelines. Lang Speech Hear Serv Sch. 2007;38(3):190–200.
  • Laing SP, Kamhi A. Alternative assessment of language and literacy in culturally and linguistically diverse populations. Lang Speech Hear Serv Sch. 2003;34(1):44–55.
  • Statistics Sweden [Internet]. Population Statistic for 2016–2020 [cited 2021 Nov 1]. Available from: http://www.scb.se.
  • Law J, McBean K, Rush R. Communication skills in a population of primary school-aged children raised in an area of pronounced social disadvantage. Int J Lang Commun Disord. 2011;46(6):657–664.
  • Hasson N, Camilleri B, Jones C, et al. Discriminating disorder from difference using dynamic assessment with bilingual children. Child Lang Teach Ther. 2013;29(1):57–75.
  • Nayeb L. Rikshandboken för barnhälsovård: Språkundersökning av flerspråkiga barn [National handbook for child health service: Language examination in bilingual children] [Internet]2019. July 3 [cited 2021 Nov 7]. Avaliable from: https://www.rikshandboken-bhv.se/metoder–riktlinjer/sprakundersokning-och-screening/sprakundersokning-av-flersprakiga-barn/#section-15743. Swedish.
  • Nayeb L, Lagerberg D, Sarkadi A, et al. Identifying language disorder in bilingual children aged 2.5 years requires screening in both languages. Acta Paediatr. 2021;110(1):265–272.