The utility of the International Classification of Functioning construct as a statistical tool – operationalizing mental health as an indicator of adolescent participation

Abstract

Background

The ICF provides a common scientific language for the study of health and functioning. Adolescent mental health, operationalized as engagement in life situations, is one aspect of functioning. Engagement as mental health has a bi-directional relation with environmental factors.

Aim

To test the statistical utility of the International Classification of Functioning (ICF) classification in coding adolescent mental health and mental health problems.

Methods

Using data measuring mental health in a representative Swedish sample of 12–13-year-olds linking responses to the classification codes. The internal structure of the classification system constructs was tested using factor analysis.

Results

A factorial solution could be found for most chapters indicating that the ICF framework and coding system could be used; however, the variance explained was quite low. Linking worked better at code-level, rather than chapter level. Items measuring risk behavior or risk factors are loaded in separate constructs.

Conclusions

When coding items for statistical purposes, code-level rather than chapter level is to be preferred. Also, participation in risk behavior loads in separate factors indicating that these behaviors are separate from other types of participation.

Implications for rehabilitation

• Considering some challenges with varying level of detail in the ICF-CY’s chapters, the framework can be used to identify the content of mental health questionnaires to be used in rehabilitation.

• To provide more detailed information in rehabilitation addressing mental health, a code-level solution is more appropriate than a chapter level solution.

• Despite the use of same ICF-CY codes, negative participation, i.e., risk behavior, measures a different dimension than positive participation, is especially relevant in rehabilitation addressing mental health.

Keywords:

• ICF
• participation
• mental health
• adolescents
• risk behaviors
• factor analysis

Introduction

Mental health is seen as having a positive influence on functioning, while mental disorders are considered pathological. By using a common framework and taxonomy for functioning, disability, and health, it is possible to investigate what aspects of everyday functioning for adolescents that are related to mental health. Stucki [1] points out that when a wide variety of measurements is used to survey functioning the measurements contain a wide variety of constructs that make measurement results difficult to compare. By translating collected data into a common framework, such as the International Classification of Functioning (ICF) [2] and its child and youth version ICF-CY classification, we can develop a common terminology to reduce potential misunderstandings. The ICF uses an alphanumeric coding system, in which letters stand for different components, the first number indicates the chapter, and subsequent numbers represent more detailed specified codes. These codes provide the content information that can be shared, even when the direct phrasing of items in instruments differs. For example, in the code “b110,” b stands for the component “Body functions,” b1 indicates the chapter “Mental functions,” and b110 tells us that the code specifically categorizes “Consciousness functions” as a part of “Global mental functions.” Such codes ought to work well statistically, as one code stands for one concept, independent of the many ways’ ideas can be phrased in different measures. These codes can be seen as latent constructs and can therefore be operationalized in different ways.

Hypothetically, using the ICF-CY [3] framework and coding system makes it possible to survey the same aspects of everyday functioning efficiently, independent of type of questionnaires used. In practice, however, this reasoning requires data collected within other frameworks of functioning to be successfully linked (aka translated), into ICF-CY codes and used to study everyday functioning within the ICF-CY framework. It is a special challenge to translate instruments aimed at measuring latent constructs in frameworks other than the ICF-CY into ICF-CY codes. For example, within one instrument, some questions might measure one aspect of an ICF construct while other questions measure another aspect. Another challenge is that some ICF chapters are broad, encompassing many different constructs, especially environmental chapters as well as the broader participatory chapters for example Community, social life, and civic life (d9). The ICF-CY model also focuses on individual functioning (mental health) rather than on deficits (mental illness or mental symptoms). In linking mental health to ICF-CY codes, common questions are: Can both good mental health and mental illness in individuals be described using codes that were primarily developed to describe functioning? Can these codes be contextualized to describe body and activity as well as participation levels? Can participation in everyday activities be seen as an indicator of mental health?

Items from extant instruments can be linked to the ICF-CY, as a way of using inferential statistics on already collected data. However, then it is necessary to use just one code per item, rather than one code for every small codable unit in an item, as recommended by established linking rules [4]. The established rules have been used successfully in studies that have descriptively investigated codes in instruments [5] or in order to compare content between instruments [6–9]. These studies all have used several codes per item; very few studies have used only one code per item. In a previous study, we critically reviewed and trialed the linking rules in relation to latent constructs, finding that it was possible to use the ICF-CY to code individual items from instruments to one code with high inter-rater reliability [10]. However, the question remains, will this type of coding work statistically, can we prove that the construct of ICF-CY still holds, when using data this way?

In the ICF-CY classification system, component activities and participation are presented in a single component,” which includes nine neutral chapters, e.g., “Communication” (chapter d3) and “Domestic life” (chapter d6). As no evident distinction between activity and participation exist within the classification, researchers must decide on beforehand how to separate activity and participation. According to the ICF, this division can be carried out in four different ways, with or without overlapping codes [11,12]. The fact that there are several different ways to separate these constructs constitutes a conceptual challenge, as acknowledged by Badley [13] and Perenboom and Chorus [14]. This challenge partly reflects the diffuse operationalization of participation. ICF-CY components differ in the precise ways in which they describe phenomena, using more detail to describe the body and less detail to describe the environment [12]. In addition, the level of detail differs between chapters within the activity/participation component, with more details included in the lower chapters. Some researchers refer to the first four chapters as activity and the latter five as participation [8,15], perhaps for this reason. Thus, it is reasonable to assume that ICF-CY chapters, dependent on the level of detail, will differ in the extent to which they have a unidimensional load in a factor analysis.

Functioning is a broad term in the ICF-CY, covering body, activity, and participation levels. When functioning is contextualized, as it is on the level of participation, the environmental factors that influence a person’s functioning must also be described. While participation is a multidimensional concept that can be operationalized in different ways [16], the ICF-CY defines participation as “engagement in life situations” [12]. In the ICF-CY, it is currently only possible to assess being there, i.e., frequency of attendance [16]. However, it is necessary to describe functioning in relation to both being there and engagement. There is an ongoing discussion regarding the difficulty involved in obtaining clear statistical factors to measure participation [17]. In a previous study, we reported the need to investigate ways in which assessment of participation based on ICF-codes can be statistically validated, using measurable factors, to further comparative research. We identified several factors that made it challenging to link mental-health-related instruments to ICF-CY codes [10]. Many of the challenges referred to maintaining the intended aim of using the instruments when coding to ICF-CY. It requires selecting coding strategies when linking to ICF that allow for keeping both the latent construct of the original instrument and the ICF codes.

The following research questions have therefore been addressed:

1. Can statistical analysis support an ICF-CY construct? – i.e., can ICF-CY as a framework be used as secondary analysis on data already collected.

2. Is the linking process statistically valid when tested using a dimension reduction technique?

3. Can the ICF-CY chapter levels be confirmed in an item reduction analysis or do item level provide a better fit?

4. How will items that measure different risk behaviors or negative participation load when dimension reduction techniques are applied to data?

Methods

Data from a questionnaire package addressing mental health in adolescents were used for the analysis. This questionnaire package was retrieved from the Swedish research programme LoRDIA (Longitudinal Research on Development in Adolescence). In LoRDIA, data were collected in four small- to medium-sized Swedish municipalities, representing the typical size of municipalities in Sweden. Ethical permission for LoRDIA research program, including the consent procedure, data collection, and the experimental procedure, was granted by the Regional Research Review Board in Gothenburg (no. 362-13; 2013-09-25).

Participants

All adolescents, in grades 6 and 7, in these four municipalities, were invited. That is a total population of 12- and 13-year-olds, including adolescents with intellectual disabilities attending special schools. The total population were N = 2018, of these 228 opted out, leaving a population of N = 1790 adolescents, of these 275 did not participate. The sample consisted of 1515 students in the 6th (51.5%) and 7th (48.5%) grades. There was a small overrepresentation of girls (51%). Due to the sampling procedure, no administrative information of diagnosis exists, only self-rated illnesses and impairments. For example, did 18% self-reported some form of neuropsychiatric disability: these included difficulties reading, writing, or calculating, ADHD, intellectual disabilities, autism spectrum disorder, and/or other psychological disorders. The numbers reported correspond to prevalence rates in Sweden [18].

Instrument

As we wanted to investigate the statistical support for coding secondary data into ICF-CY codes, the entire questionnaire package was coded into ICF-CY codes. The package, consisting of 25 different questionnaires incorporating multiple scales, was distributed to students [19]. The scales used in the package focused on broad aspects of adolescent everyday life. The questionnaires are presented in Supplementary Material.

Procedure

Two independent coders (LA and FL) linked the questionnaire items to ICF-CY codes. Coding was based on established linking rules [4,20] and instructions given in ICF-CY [12,p.249]. It was carried out in accordance with linking rules refined in 2016 [21]. Meaningful concepts within each item were identified in relation to the latent constructs of the original questionnaire and the most precise code was chosen. The underlying concepts were discussed, and additional independent coders were used when needed, see Augustine et al. [10]. The aim was to link concepts to the ICF-CY codes in a way that reflected the items’ essential meaning (latent construct), rather than their wording. During the linking process, most items were given second level ICF-CY codes. The inter-rater reliability was sufficient, with κ = 0.734 [10].

Analysis

Focus for this paper, was only to investigate the statistical support for using an ICF-CY linking approach on data already collected. To test the applicability of the ICF-CY coding construct for these questionnaires, factor analyses were conducted on chapter level. A total of 11 factor analyses were conducted, one for body function, seven for activity/participation and three for the environmental chapter. Although the first level codes (chapter level) were used for the factor analyses, information about second level classifications existed at item level. Making it possible to analyze factor content based both on second level ICF-CY classifications and in comparison to the original scales for the items.

In the factor analyses, we used correlations rather than covariation, as variables could have different numbers of responses. Maximum likelihood was chosen as a method of extraction and a direct oblimin rotation was used to determine factors. To test how suitable the data were for factor analysis, we used the Kaiser–Meyer–Olkin (KMO) test; KMO values of less than 0.6 were deemed inadequate. In addition, Bartlett’s test of sphericity was used to determine redundancy, indicating that variables could be summarized to produce fewer factors. The Kaiser criteria for factors with an Eigen value over 1 were used to determine the number of factors extracted. This was completed using Cattell’s scree test. In addition, factors that explained less than 3% of variance were excluded. Looking at individual items within the analysis, items with too-low communalities were dropped; ideally, all communalities were expected to be over 0.30. We settled for 0.20 to ensure that as few chapters as possible lacked a factor solution or had a very small number of items.

Results

We wanted to investigate if there was statistical support for using the ICF-CY constructs on secondary data measuring mental health, and factors related to mental health. We found that despite using a large questionnaire package, items did not cover all ICF-CY chapters. Also, some chapters were over-represented, i.e., there were more items relating to these chapters than what we needed for creating indexes. For example, chapter d1 (Learning and applying knowledge) was not represented at all, Self-care (d5) was only represented by items measuring risk behaviors. Some chapters such as d6 (Domestic life) and d8 (Major life areas) were represented by only four items. While chapters regarding b1 (Mental function) and e3 (Social support) and e4 (Attitudes) were overrepresented. These differences between components and chapters in what items represented made it hard to statistically analyze the entire ICF-CY construct in testing its utility for mapping mental health data.

To respond to our two research questions, regarding whether linking to ICF-CY codes can be used for statistical analyses of mental health, a key issue was if we can create ICF-CY constructs on chapter level and if the created constructs were statistically valid. Therefore, factor analyses were applied on data (see Table 1). For most chapters, more than one factor was identified for items. In most cases, specific items within the chapter were central for the factors identified.

Table 1. Factorial loadings and factors, investigating the structure of coding variables into ICF-CY.

ICF-CY chapters	No. of variables	KMO	Total variance explained	Rotation sums of square loadings^a	% of variance	Identified factor
Mental functions (b1)	35	0.875	30	4.6	15.9	Emotional functions (b152)
				2.8	7.3	Temperament and personality (b126)
				1.7	4.2	Experiences of self and time functions (b180).
				4.5	2.6	Psychosomatic complaints (b152 and b280)
General tasks and demands (d2)	17	0.78	30.5	2.6	17.0	Managing one’s own behavior (d250)
				1.8	7.0	Handling stress (d240)
				1.7	6.5	Managing one’s own behavior (risk behavior) (d250)
Communication (d3)	8	0.613	33.6	1.76	24.1	Conversations and discussions (d350–d355)
				1.38	9.6	Sharing information at home (D350)
Self-care (d5)	7	0.609	50.5	1.8	30.4	Both measuring
				1.9	20.1	Looking after one’s own health, negative (d570)
Domestic life (d6)	4	0.703	30.8	–	30.8	Importance of household chores (d630–d649)
Interpersonal interactions and relationships (d7)	18	0.725	47.0	1.6	9.4	Negative relationship with a stranger (d730)
				2.5	14.4	Informal social relations, conflict (d710/d750)
				1.9	11.6	Boyfriend/girlfriend (d770)
				2.6	12.8	Sibling relationship (d7602)
Major life areas (d8)	4	0.715	32.52	1.30	32.52	School education (d820)
Community, social and civic life (d9)	9	0.500	–	–		No fit
Products and technology (e1)	7	0.742	36.30	1.79	25.63	Products for personal use (e110)
				0.73	10.40	Assets (e165)
Support and relationships (e3)	43	0.799	30.443	7.57	14.82	Support from immediate family – parents not listening (e310)
				9.03	4.61	Support from immediate family – parental warmth (e310)
				4.99	3.84	Support from people in positions of authority (teacher) (e330)
				4.03	2.81	Support from immediate family – parental angry outburst or rejection (e310)
				4.04	2.24	Support from immediate family – siblings (e310)
				6.61	2.13	Support from immediate family – attachment (e310)
Attitudes (e4)	31	0.843	27.26	4.07	12.5	Societal attitudes, in particular negative ones related to bullying and harassment: negative (e460)
				3.15	10.2	Attitudes of immediate family; parental control and knowledge (e410)

^aKaiser rule Eigenvalue > 1. Higher values explains more of variance.

In most chapters, valid factor models were created. However, these constructs were more coherent on item level than on chapter level. Also, within the processes of item-reduction many variables needed to be excluded, due of bad fit. It led to that representation was lost within chapters. The variance explained in the analyses on chapter level were low, usually around 30–35%: chapters represented by many items, allowed construction of scales on item level which explained more variance (see Table 1). In most chapters, some items formed factors, except for Community, social life, and civic life (d9). Most factors match second-level codes within the given chapter, i.e., on a three-digit level, supporting the construct of ICF-CY as a mean of secondary analysis. Chapters conceptually broad such as Community, social and civic rights (d 9) cover aspects that are too diverse to support the construction of factors. In Social support (e3), other broad contextual factors, many linked items from the instruments received the same code (e310) needed to be differentiated more than what was possible using the three digits allowed by the linking rules.

When analyzing participation in risk behaviors linked to ICF-CY codes, they load in separate factors within ICF-CY chapters. For example, in General tasks and demands (d2), there are two factors relating to Managing one’s own behavior (d250), one relating to positive participation and one to risk behaviors. All items coded as Self-care related to risk behavior and hence loaded in the same construct. Also, in Interpersonal interactions and relationships (d7), a specific factor for participation in risk behaviors was identified. In the context of mental health, there seems to be a division on item level relating to positive and negative aspects of functioning in the environmental codes. This fit with the notion of barriers and facilitators within the ICF-CY. We find these barriers in Support and relationships (e3) as parents not listening or having angry outbursts, and regarding Attitudes (e4) relating to harassment and bullying. Risk-factors or participation in risk activities is something else than just the negative spectrum of more neutral or prosocial aspects of functioning and needs to be handled as such.

Discussion

ICF-CY measures participation in life-situations, something that can be seen as health and well-being, analyzing data related to mental health is therefore relevant. Conceptual challenges exist when applying statistical analyses to mental-health-data linked to ICF-CY codes. The present study uses analytic statistics to “build” mental health factors based on ICF-CY chapters. Our study partly support that the ICF-CY construction can be used. However, it must be emphasized that when used for comparative statistical use, especially using secondary data that did not primarily try to use the ICF-CY framework, second level codes work better than chapter level, i.e., item level. If chapter levels still are used, it must be clear what specific three-digit codes that are put to represent the entire chapter. Also, our results support that we can create a statistically valid constructs even though the total variance explained is low. The result indicates that the theoretical interest of using the ICF-CY model weight heavily when decisions are made about what model to use.

Based on the same data set, a previous study argued that linking is possible and that most ICF-CY areas relevant to mental health were covered, and that the measures of youth mental health used in LoRDIA capture multidimensional aspects of functioning [10]. However, this study shows that some chapters within ICF-CY relevant for mental health are not covered. No questions about Learning and applying knowledge (d1) were found in this extensive questionnaire; and there were only two items about Mobility (d4). Learning and applying knowledge and Mobility are aspects of functioning that we know are related to mental health [22].

Arguably, using a whole battery of questionnaires makes it necessary to reduce the number of items. The decision to include all items were chosen to test whether any underlying construct fits the ICF-CY coding framework. Initially, we included all possible items in the factor analysis, but in each chapter, items needed to be dropped to create a fit. Depending on both the construct and the quality of items, different numbers of items were dropped. For chapters measuring more complex aspects of activity and participation, such as Interpersonal interactions and relationships (d7), Major life areas (d8), and Community, social and civic life (d9) this was more challenging. In all, more than half of the variables were dropped due to low variability, low communalities, or low anti-image correlation. No factor solution was found for the chapter Community, social and civic life (d9) despite this. Overall, the results of the current study indicate that, if contextualized in everyday activities, most chapters can be represented by indexes that can be used statistically. The items ought to load in just one factor to be used at chapter level. This happens when there are few variables and all including items represent one or few ICF-CY codes, for example as seen in the case of Domestic life (d6). Including second-level codes usually generates multiple factors within the same three-digit code; Support and relationships (e3) provides a good example. Most items and five of the six support factors measured Family support (e310), indicating that a third level of four-digit codes would be needed to differentiate more clearly between environmental codes.

When validating the linking process using factor analyses, the results showed that our chapter operationalization did not work as well as using second-level codes, i.e., chapter level, for identifying factors. When second level codes, i.e., three-digit ICF-CY codes, are used it excludes other possible aspects still present at chapter level. However, these second-level codes work better from a statistical point of view, as they represent a more well-defined aspect of functioning. Another solution could be to use one or a few specific codes to represent a chapter, as we see in high-alpha subscales that primarily measure one second-level code. Some challenges can be managed by using data obtained from other studies within an ICF-CY framework where the original scales load in factors of their own. For example, family support (e310) items tend to load in specific factors and therefore a whole scale can be linked to one code instead of mixing different scales to measure the same code. It is possible that more scales need to be constructed that aim to represent single second level codes in ICF-CY.

One concern, when coding mental health items – given that mental illness often relates to behavioral difficulties in children was that these risk factors would become invisible because the participation codes are neutral. The present study supports the argument that mental health and mental illness are different constructs and not just different ends on the same continuum [23]. It indicates that participation (in neutral terms) is directly related to mental health but maybe indirectly to mental health problems [24]. In this study, risk items seemed to load in their own underlying constructs. Risk behaviors are not prevalent in our material, perhaps due to the age of the study sample (12–13 years). Many items were excluded due to low variability; few students rated themselves as engaging in various risk behaviors. Yet we clearly see that items related to risk behavior and positive participation respectively measure different underlying dimensions. This is visible in several of the factor analyses. Measuring risk behavior is different from measuring participation in pro-social activities, even if they are coded using the same ICF-CY code. The present study identified four such risk participation scales: the risk factors for future mental health, Handling demands and stress (d2), Looking after one’s health (e5), and negative Interpersonal interactions (d7). Two environmental risk factors were also identified, negative Attitudes (in this case, bullying) (e4), and angry parental outbursts and rejection in Support and personal relationships (e3).

Participation is a multidimensional concept, and this was visible in the factor analysis. In the present study, it was easier to construct subscales for the early chapters in the activity/participation component (up to chapter d5). The difficulties involved in creating subscales for participation in ICF-CY chapters d6–d9 may reflect a lack of theoretical understanding of participation, prior to linking items to ICF-CY. For example, some items are designed to measure frequency of participation, something that is possible in the ICF-CY at present. Other items are designed to measure the degree of involvement or the importance of participation, something that is not possible in the ICF-CY [25]. Only items that measured importance remained in the factor solution for Domestic life (d6), while a distinction was visible between different qualifiers of participation in chapters d7 to d9. This indicates that frequency and importance measure different aspects. As of today, the ICF-CY does not contain any qualifier for assessing engagement or the experience of involvement [16].

The aim of linking existing data to ICF-CY codes in this study was to use information to analyze mental health data statistically. In linking, the most precise code should be used and interpreted in relation to the study purpose [9]. This can be a difficult task when it comes to latent variables, especially when higher order levels of functioning are assessed. In a published recommendation on how to handle pre-existing data retroactively (i.e., the draft version of the WHO practical manual for using the ICF) [26], some solutions are recommended: “This may be done via a process of mapping or linking whereby the high level concepts or components of measures … are mapped or linked to ICF components” [p.13]. As experienced in this study, this recommendation might lead to that indexes used in the ICF-CY framework, based on a restricted number of variables, to represent the entire chapter in an ICF-CY framework. Such reductions of the number of items will create a better factorial fit, especially in higher order chapters, such as d7 to d9, and when using the environmental chapters. Whether such chapter indexes can represent a whole chapter need to be investigated in future research.

Limitations of the study

When linking items from a questionnaire designed to assess mental health in adolescents to the ICF-CY framework, some difficulties are inherited from the original scales. For example, items with underlying meaningful concepts may have been vaguely described. This makes it difficult to agree on how items should be interpreted by participants and coders. Such difficulties also generate statistical difficulties. Still, linking is worthwhile, considering that the universal language of ICF-CY offers the possibility of studying health as functioning. Although we coded all variables within a dataset, this is not necessary nor practical when comparing different data sets with the help of ICF-CY codes and chapters. Reducing the number of items in a way that captures relevant concepts is recommended. It was sometimes problematic to include several items from a single chapter because of that they represented diverse concepts.

Concerning mental health in adolescents, the chapter Learning and applying knowledge in school (d1) did not appear when the mental health of children and young people was measured in the LoRDIA material. However, learning in school is a key issue in mental health [22]. In ICF-CY, the only ICF-CY codes related to school subjects are reading, writing and mathematics, in Learning and applying knowledge (d1). Furthermore, codes for Self-care (d5), were only represented by managing one’s own health. Other aspects of Self-care, such as personal hygiene, may also be a vital part of functioning when assessing mental health in adolescence. Items covering these aspects of functioning would improve the relevance of assessing activity and participation in everyday activities as an indicator of mental health.

Implications

The results of this study indicate that existing mental health questionnaires can be linked to ICF-CY codes. This makes it possible to use the structure of the ICF-CY framework and the common language this framework provides. However, some challenges must be considered. ICF-CY chapters with broad definitions of mental health must be discussed to ascertain which chapter codes that must be represented by items with a mental health focus. Hence, only the overarching latent construct of an item tells us how to measure participation in a specific activity. In addition, a coherent scale construct must be added, with qualifiers focusing on being there as well as involvement when attending/being there.

The present study also demonstrates the challenges involved in creating statistically valid indexes based on item reduction for environmental factors as participation is intertwined with environment. Therefore, the perspective taken in the linking procedure should be determined clearly in a pre-defined coding scheme in line with the recommendations proposed in our previous published article describing challenges with the linking procedure [10]. Environmental factors in the ICF-CY lack specificity. For example, there are many aspects of support. The aim in linking environmental information to ICF-CY codes must be specified before creating indexes [10]. Fewer third level ICF-CY categories relate to environmental chapters than to the other components. New third level codes may have to be created when working with environmental chapters. From a conceptual point of view, having more third-level specific codes in Attitudes (e4) could support the construction of indexes related to mental health.

Participation in so-called risk behavior is different from other types of participation in ICF-CY chapters because it measures an aspect other than mental health. Activity and participation codes have a positive valence in ICF-CY. Participation is a neutral term; qualifiers are used to determine whether or not there are restrictions [12]. Arguably, mental health requires one core set of codes, while mental illnesses and diseases require others. As participation in risk activities loads in separate subscales, mental health may be differentiated from mental illness. In addition, assessing mental health probably requires another selection of codes to represent each chapter within the ICF-CY then codes used to analyze the needs of people with chronical physical illnesses. Selection of codes may be supported by reviewing core sets for different diagnoses (see for e.g., [27,28]). We recommend that participation in risk activities should be categorized as participation in deviant activities, or a negative form of participation, using extant ICF-CY codes. In measuring mental health, participation in risk activities, such as drinking, smoking, stealing, and truancy, can be classified as participation if operationalized as frequency of attending and/or intensity of involvement.

Disclosure statement

The authors report no conflicts of interest.