Psychometric properties of the Teenage Executive Functioning Inventory (TEXI): A freely available questionnaire for assessing deficits in working memory and inhibition among adolescents

ABSTRACT

The Childhood Executive Functioning Inventory (CHEXI) and the Adult Executive Functioning Inventory (ADEXI) are reliable and valid rating instruments for assessing working memory and inhibitory control. However, a teenage version of this instrument has not been available, and the aim of the present study was therefore to investigate the psychometric properties of the Teenage Executive Functioning Inventory (TEXI). After interviewing both researchers/clinicians and adolescents themselves, a questionnaire with 20 items was created. Data from adolescents age 13–19 years (n = 302) and their parents were thereafter collected. Factor analysis showed that the TEXI has two clear factors: working memory and inhibition. Further, the TEXI was shown to have high reliability in terms of internal consistency (≥ .85), split-half reliability (≥ .81) and inter-rater reliability between self-ratings and parent ratings (.82). In sum, the TEXI is a reliable questionnaire for measuring working memory and inhibition in adolescents. The two versions of the TEXI are presented in the appendices of this paper and they will be freely available on the Internet.

KEYWORDS:

• Executive functioning
• working memory
• inhibition
• psychometric properties
• rating instrumnet

Executive functioning (EF) is defined as complex cognitive processes necessary to maintain goal‐directed behavior such as working memory, inhibition, set‐shifting, and planning (e.g., Welsh & Pennington, 1988) and EF has been shown to be related to many important outcomes in daily life (review by Diamond, 2013). A large body of research has therefore focused on how to best assess EF deficits. The most common methods include different types of laboratory tasks. However, in recent years, there has been an increased interest in the development of EF rating instruments. One reason for this is that ratings have been shown to be more strongly related to daily life functioning compared to tests (e.g., Barkley & Fischer, 2011; Barkley & Murphy, 2010). In addition, ratings have the advantage of being easier to use when screening for EF deficits in a large number of individuals. However, previous research has also shown that it is much more difficult to distinguish between different neuropsychological deficits with ratings than with tests (e.g., Sjöwall & Thorell, 2019), and ratings are always affected by biases. Thus, ratings should preferably be used as a complement to, rather than as a replacement for, tests (Toplak, West, & Stanovich, 2013).

Unfortunately, many available instruments for assessing EF deficits suffer from important limitations. As described in more detail below, most available instruments 1) do not focus specifically on executive functioning, 2) are long and complex instruments, 3) are protected by copyright and are therefore costly to use. In order to address these limitations, the Childhood Executive Functioning Inventory (CHEXI; Thorell & Catale, 2014; Thorell & Nyberg, 2008) and the Adult Executive Functioning Inventory (ADEXI; Holst & Thorell, 2018) were created. However, neither one of these two instruments is suitable for use in adolescence, and the aim of the present study was therefore to introduce the Teenage Executive Functioning Inventory (TEXI). Previous research has shown that both the CHEXI and the ADEXI are reliable and valid instruments and they have been shown to be related to important outcomes such as academic achievement in childhood (Thorell, Veleiro, Siu, & Mohammadi, 2013) and daily life functioning in adulthood (Holst & Thorell, in press). Several previous studies have also shown that the CHEXI and the ADEXI can differentiate well between individuals with Attention Deficit Hyperactivity Disorder (ADHD) and controls without a psychiatric disorder (Catale, Meulemans, & Thorell, 2015; Holst & Thorell, 2018; Thorell, Eninger, Brocki, & Bohlin, 2010; Thorell et al., 2017). One important advantage of the CHEXI and the ADEXI is that these rating instruments focus specifically on deficits in working memory and inhibition, without including items that are more or less identical with the symptom criteria for ADHD (e.g., “is impulsive” and “has a short attention span” are included in available instruments as examples of items measuring inhibition and working memory). The CHEXI and the ADEXI also have the advantage of being relatively short instruments, and they are freely available for download in several languages besides English, such as French, German, Dutch, Swedish, Danish, Italian, Spanish (including a Spanish-Latino version, Catalan, and Galician), Chinese, Japanese, and Persian.

It has been argued that poor executive functioning has increased negative impact in daily life during adolescence, as this period in life demands increased responsibilities both at school and in relations with peers (e.g., Burnett, Scratch, & Anderson, 2013). However, there are still relatively few reliable EF ratings that have been created especially for use with adolescents, especially instruments that focus specifically on EF deficits rather than including related constructs such as ADHD symptom levels. In addition, it has been emphasized that we need to interpret the variability between informants as clinically meaningful information (Dirks, De Los Reyes, Briggs-Gowan, Cella, & Wakschlag, 2012). It should therefore be considered important to include both self-ratings and informant ratings.

Based on the above-described limitations of available instruments for assessment of EF deficits in adolescence, the aim of the present study was to investigate the psychometric properties of the Teenage Executive Functioning Inventory (TEXI). Like the ADEXI, the TEXI includes both a self-rating and an other-rating, and it will soon be freely available on the Internet in as many languages as possible.

Methods

The present study included 302 adolescents (37% boys) age 13–19 years (M = 16.40, SD = 1.76) from four elementary (33%) and three high (67%) schools. Participants were recruited from schools in Republic of Serbia, with schools located in larger cities (43%), smaller cities (25%) as well as in rural areas (32%). Most of the participants lived with both parents (73%). Most parents had completed secondary school (69%), with a smaller number of them having only completed elementary school (8%) or having a university education (23%). These characteristics of our sample are representative of the total population according to official data from the Statistical Office of the Republic of Serbia (2019). After giving their informed consent for participation, both parents and adolescents completed the TEXI in the home setting. The participants received no compensation for taking part in the study. The study was conducted in accordance with the ethical guidelines in the country in which the data were collected.

When developing the TEXI, individual interviews with researchers studying adolescents and clinicians working within child and adolescent psychiatry were first conducted. The interviewees were first asked whether they thought the items included in the CHEXI and the ADEXI were relevant for adolescents. In addition, they were asked to suggest new items that were not included in the child and adult version of the questionnaire. This resulted in a teenage version that included all 14 items included in the ADEXI. In addition, three items from the CHEXI were included (i.e., difficulties stopping an activity even when told to do so, having problems motivating oneself to do boring things, and getting more excited compared to one’s peers). Finally, the following three new items were added, as they were considered to cover executive skills of great importance to adolescents’ daily life functioning: 1) starting a task when distracted by something more fun, 2) failure to finish a task one has started, and 3) putting things off until the last minute. All items were thereafter revised to make the sentence structure and choice of words as simple and clear as possible for adolescents. As part of this process, twenty adolescents age 13–15 years completed a preliminary version of the TEXI and provided feedback on how the items could be revised. The final version of the two TEXI versions (i.e., self rating and parent/teacher rating) are presented in the supplementary material. The CHEXI and the ADEXI are freely available on the internet (www.chexi.se) in several different language versions and our plan is to also make the TEXI freely available in as many langauges as possible. Similarly to the CHEXI and the ADEXI, the TEXI uses a 5-point likert scale ranging from 1 (“definitely not true”) to 5 (“definitely true”)

With regard to statistical analyses, exploratory factor analyses were first conducted with separate analyses for parent ratings and self-reports. To study reliability, internal consistency was examined using Cronbach’s alpha. In addition, split-half reliability was examined using the Spearman-Brown coefficient, and intraclass correlations (ICC; two-way random effects model, absolute agreement) were used to investigate interrater reliability between self-ratings and parent ratings. Finally, paired t-tests were used to investigate the extent to which self‐ratings and parent ratings differed significantly from one another within individuals. Raw scores were used in all analyses.

Results and discussion

Factor analysis

The Kaiser-Meyer-Olkin measure of sampling adequacy was .92 for parent ratings and .90 for self-reports. Further, Bartlett’s test of sphericity was significant for both parent ratings and self-reports, both χ² = 2360.81, p < .0001. For all items included in the TEXI, the scores varied across the whole range (i.e., from 1 to 5) for both parent ratings and self-reports. Thus, the data appeared to be suitable for conducting factor analyses. The results of the factor analyses showed that three factors with eigenvalues above 1.0 emerged for parent ratings and four factors for self-reports. Neither of these solutions appeared to fit the data very well, with several items loading higher than .30 on several factors. Several double loadings were also found when exploring a 3-factor solution for the self-ratings. However, as shown in Table 1, a 2-factor solution showed two clear factors, explaining 51.73% of the variance in parent ratings and 45.05% in self-reports. An oblique rotation method was chosen, as two factors were shown to be highly correlated, r(300) = .51 (parent ratings) and r(300) = .50 (self-reports). One factor was comprised of items tapping into inhibition and the other factor was comprised of items tapping into working memory. The three new items not included in either the CHEXI or the ADEXI (i.e., initiating a task, finishing what one has started and procrastination) were all shown to be part of the inhibition factor. These results were similar to those for the ADEXI and the CHEXI, as well as a theoretical model suggesting that working memory and inhibition are two major aspects of executive functioning (e.g., Diamond, 2013). When creating the CHEXI, four a priori factors were presented (i.e., working memory, planning, inhibition and self-regulation). However, the original factor analysis for the CHEXI (Thorell & Nyberg, 2008) showed that a two-factor solution fit the data better, and the results of the present study show that the same factor structure should be applied for the TEXI. Thus, the obtained two-factor solution obtained for the TEXI meant that 1) items related to memory and planning belonged to the same factor and 2) the inhibition factor included not only items related to the ability to stop an ongoing activity, but also a wider range of self-regulatory skills such as “acting livelier/wilder compared to peers” and “refraining from laughing in situations where it is inappropriate.” The three newly added items all loaded onto the inhibition factor, which is in line with previous research showing an association between procrastination and inhibitory control (Steel, 2007).

Table 1. Factor structure of the Teenage Executive Function Inventory (TEXI). Items sorted according to how highly they load on each factor for parent ratings.

	Parent report		Self-ratings
	INHIB	WM	INHIB	WM
10. Has difficulties stopping him-/herself during an activity he/she likes (e.g., sits in front of the computer/mobile device even though it is time to go to bed) late in the evening even though I know I should go to bed).	.82		.81
16. Has difficulties starting a task if distracted by something he/she likes (e.g., fails to start doing homework and instead uses his/her mobile device)	.78		.82
15. Has difficulties motivating him-/herself to do things that he/she does not like to do	.75		.73
20. Puts things off until the last minute	.68		.50
6. Sometimes has difficulties refraining from laughing or smiling in situations where it is inappropriate	.67		.65
3. Has a tendency to do things without first thinking about what could happen	.63		.49
17. Has difficulties stopping an activity when asked to do so	.62		.68
4. Has difficulties stopping an activity even though he/she is told that it is not allowed	.60		.61
14. Appears to be livelier/wilder compared to his/her peers	.53		.54
19. Fails to finish things that he/she has started	.51		.40
18. Often gets more stoked (excited) compared to his/her peers if something special happens (e.g., parties, trips, birthdays, winning a computer game)	.39		.54
11. Sometimes has difficulties understanding instructions unless he/she is also shown how to do something		.86		.56
12. Has difficulties with tasks involving several steps that need to be completed in a certain order		.78		.70
1. Has difficulties remembering lengthy instructions		.75		.58
5. When someone asks him/her to do several things, he/she sometimes cannot remember all of them		.75		.72
7. Has difficulties coming up with a new way to solve a problem when he/she gets stuck		.74		.75
8. When asked to get something, he/she sometimes forgets what he/she is supposed to get		.70		.69
2. Sometimes has difficulties remembering what he/she needs to do in the middle of an activity		.68		.74
9. Finds it difficult to plan things (e.g., remembering to bring everything necessary for school or when going on a trip) trip/to work/to school)		.54		.59
13. Has difficulties learning from his/her own mistakes (e.g., repeats the same mistake over and over again)		.45		.47
Explained variance (%)	41.55	10.18	35.73	9.32

Factor loadings lower than .30 are not displayed in the table. INHIB = Inhibition and WM = Working memory.

Associations with age and sex

Table 2 presents means, standard deviations, skewness and kurtosis. None of the subscales for either parent ratings or self-reports were significantly related to age, all rs (300) < .09. Regarding sex differences, boys (inhibition: M = 2.84, SD = .84; working memory: M = 2.10, SD = .85) and girls (inhibition: M = 2.70, SD = .85; working memory: M = 1.99, SD = .70) did not differ significantly for parent ratings, both ts(300)<1.41. For self-ratings, the effect was significant for working memory, t(300) = 2.24, p < .05) and a marginally significant for inhibition, t(300) = 1.94, p = .053. Boys (working memory: M = 2.29, SD = .83; inhibition: M = 3.11, SD = .78) had higher executive deficits compared to girls (working memory: M = 2.09, SD = .66; inhibition: M = 2.93, SD = .82). However, it should be noted that the effect sizes were small even for the significant or marginally significant effects (d = .27 for working memory and d = .22 for inhibition). It is therefore unlikely that these very small differences have any real-life implications as an effect size needs to be at least of medium size to be “visable to the naked eye of a careful observer” (Cohen, 1992, p. 156). This finding is in line with a previous study of parent reports using the CHEXI in Swedish and Spanish samples, which showed that boys had somewhat higher executive deficits compared to girls, although sex differences were small (Thorell et al., 2013).

Table 2. Descriptive data and results of the reliability analyses.

	Parent ratings	Self-reports
Means and standard deviations
Inhibition, M (SD)	2.75 (.85)	3.00 (.81)
Working memory, M (SD)	2.03 (.76)	2.16 (.73)
Skewness (SE)
Inhibition	.08	−.13
Working memory	.77	.57
Kurtosis (SE)
Inhibition	−.71	−.45
Working memory	.09	−.13
Internal consistency (Cronbach’s α)
Inhibition	.89	.86
Working memory	.90	.85
Split-half reliability (Spearman-Brown)
Inhibition	.90	.87
Working memory	.89	.81

Reliability

The results for reliability analyses are presented in Table 2. They showed that the internal consistency for the two factors was good to excellent, with Cronbach’s alpha values of .89 (inhibition) and .90 (working memory) for parent ratings and .86 (inhibition) and .85 (working memory) for self-ratings. In addition, the Spearman-Brown coefficients ranged from .81 to .90, indicating high split-half reliability. High internal consistency has also been obtained for both the CHEXI (Thorell & Nyberg, 2008) and the ADEXI (Holst & Thorell, 2018).

With regard to inter-rater reliability as measured by intraclass correlations, the results showed excellent inter-rater reliability between parent ratings and self-reports for both inhibition (.82) and working memory (.82). These coefficients are higher than those previously obtained for the ADEXI (Holst & Thorell, 2018) as well as those found for other ratings of executive function deficits such as the BDEFS (Barkley, 2011) and the BRIEF (Roth, Isquith, & Gioia, 2005). In addition to studying correlations between self‐ratings and those obtained by a significant other, it may also be of value to study mean differences. In the present study, the results showed that the scores obtained for the self‐ratings were significantly higher (i.e., larger deficits) than those obtained from a significant other for both inhibition, t(301) = 6.82, p < .001, and working memory, t(301) = 4.00, p < .001. This finding is in line with previous studies of EF ratings in adults using either the ADEXI (Holst & Thorell, 2018) or the BRIEF‐A (Roth et al., 2005), as well as with a previous study of behavior problems in adolescents (Berger, Jodl, Allen, & Davidsson, 2005).

Conclusions, limitations and future directions

In conclusion, the psychometric properties of the TEXI are similar to or better than those obtained for the childhood version (i.e., CHEXI), adult version (i.e., ADEXI), and other EF rating instruments. One limitation of the present study is that validity was not examined. It would therefore be of great value if future research were to investigate the relation between the TEXI and other self-rating instruments of executive functioning as well as associations between the TEXI and laboratory measures of working memory and inhibition. Previous studies have generally shown only modest associations between ratings and tests (Toplak et al., 2013). Rather than interpreting this as poor validity, it has been argued that ratings and tests capture at least partly different constructs, and that ratings should therefore be seen as an important complement to rather than a replacement for tests. It should also be noted that recent advances have been made using virtual reality techniques for measuring cognitive functioning (e.g., Climent et al., in press) and it will be important for future research to determine how such methods can be used in combination with ratings to achieve more ecologically valid assessments.

Another interesting finding of the present study was that the mean values for both working memory and inhibition were significantly higher for self-ratings than for parent ratings. It is beyond the scope of this study to speculate regarding possible reasons for why the adolescents reported higher mean levels of both working memory and inhibition compared to parents. However, our findings emphasize the need to include multiple raters in order to obtain a comprehensive picture of a child’s strengths and difficulties. Interestingly, Kooij et al. (2008) concluded that although patients rated their own symptom levels higher than significant others did, both tended to under‐report symptom levels in relation to investigator reports. It would therefore be of value if future research were to also obtain teacher ratings for the TEXI or observations of adolescents’ executive deficits and compare these with TEXI self-reports or TEXI parent ratings.

Disclosure statement

No potential conflict of interest was reported by the authors.

Supplementary material

Supplemental data for this article can be accessed here.

Additional information

Funding

This work was supported by the Serbian Ministry of Education, Science, and Technological Development [179006];Vetenskapsrådet [2017-01508].