Validation of Fatigue Impact Scale with various item sets – a Rasch analysis

Abstract

Purpose: Fatigue is a symptom in patients with chronic gastrointestinal (GI) and liver diseases. Different instruments have been developed to assess the severity of fatigue and the 40-item Fatigue Impact Scale (FIS) is among the most widely used. Shorter versions of FIS include the 21-item Modified Fatigue Impact Scale (MFIS), and an eight-item version for everyday use. The study aimed to assess construct validity, reliability, and sufficiency of the raw score of the original FIS with 40 items, and examine the sufficiency of the 21 items from the Modified scale and the eight items of the Daily Fatigue Impact Scale (D-FIS), all of which are embedded in the 40-item scale.

Methods: Patients with chronic GI or liver disease (n = 354) completed the FIS with 40 items. The majority (57%) was under the age of 55 years and approximately half were females (48%). Various item sets of FIS were derived which showed fit to the Rasch model.

Results: Local dependency and multidimensionality in FIS and the 21-item Modified scale were resolved with a testlet solution but the D-FIS showed local dependency and multidimensionality and differential item functioning (DIF) still remained. Two new item sets fulfilling unidimensionality and no DIF are suggested, one with 15 items and a six-item scale for daily use. The transformation table shows score-interval scale estimates for all these item sets.

Conclusions: Both the FIS and the Modified scale can be used to measure fatigue albeit requiring some adjustment for DIF. The eight-item D-FIS is more problematic, and its summed score is not valid. Alternative 15- and 6-item versions presented in this paper can offer valid summed scores, and the transformation table allows transformation of raw scores and comparisons across all versions.

Implications for rehabilitation

• The Fatigue Impact Scale and the Modified Fatigue Impact Scale can be used to measure fatigue after adjustments for differential item functioning.

• Alternative 15- and 6-item versions of Fatigue Impact Scale offer valid summed scores. The summed score for the Daily Fatigue Impact Scale is not valid.

• A transformation table with raw scores and Rasch transformed interval scale metric makes it possible to compare scores derived from the Fatigue Impact Scale, the Modified Fatigue Impact Scale and the proposed 15- and 6-item versions of Fatigue Impact Scale for research and/or clinical use.

Keywords:

• Fatigue Impact Scale (FIS)
• MFIS
• D-FIS
• Rasch analysis

Introduction

Fatigue is a common symptom in patients with long-term conditions, including chronic gastrointestinal (GI) [1] and liver diseases [2]. The understanding of fatigue in these disorders is incomplete, but fatigue is considered by many patients to be one of their most debilitating symptoms [3–7]. Within gastroenterology, it is well-known that fatigue is prominent in patients with irritable bowel syndrome (IBS) [8,9], inflammatory bowel disease (IBS) [10] and celiac disease [11], and within hepatology, primary biliary cirrhosis [4,5], nonalcoholic fatty liver disease [12,13], and hepatitis C [14].

Different instruments have been developed to assess the severity of fatigue and its impact on daily life [15,16], and in gastroenterology and hepatology, the Fatigue Impact Scale (FIS) [17] is among the most widely used. It has several versions including the original 40-item scale, a 21-item Modified scale (MFIS), and a short eight-item version for everyday use (D-FIS) [18].

Many studies have looked at the reliability and validity of the different versions of the FIS from a classical psychometric perspective, across a variety of diagnostic groups [19–23]. Modern test theory with Rasch analysis is now widely applied as the quality standard in health sciences in order to get more detailed information about dimensionality of a scale, scale validity and reliability, and sufficiency of using a summed score [24–27]. With Rasch analysis, it is also possible to transform ordinal scale scores into an interval metric [28]. To date, few studies have examined the various scale properties from a modern psychometric perspective, and in a limited number of diagnostic groups [29].

The current study aimed to assess construct validity, reliability, and sufficiency of the raw score of the original FIS with 40 items, and likewise for the 21 items of the MFIS and the eight items of the D-FIS, all of which are embedded in the 40-item scale, using a modern psychometric approach.

Methods

Participants and setting

Data were obtained from two study cohorts from the Gastroenterology and Hepatology Research Unit at Sahlgrenska University Hospital, Gothenburg, Sweden: (1) a cohort of patients fulfilling the Rome III criteria for IBS [30], who participated in a study between 2011 and 2013 assessing the pathophysiology of IBS, including investigations of GI motility, sensitivity and low-grade inflammation of the GI tract [31]; (2) a cohort of patients included between 2010 and 2013 with non-advanced chronic liver disease (Child-Pugh A), equally divided between five diagnoses: primary biliary cirrhosis, primary sclerosing cholangitis, autoimmune hepatitis, chronic hepatitis B and C. These subjects were included in a study to assess mechanisms behind fatigue in chronic liver disease [32]. Subjects in both cohorts completed paper questionnaires during a visit at the research unit specifically for participation in the studies. Characteristics of study participants are shown in Table 1.

Table 1. Characteristics of study population, n = 354.

	N (%)
Age, years, n = 350
≤54	201 (57.4)
55–69	111 (31.7)
≥70	38 (10.9)
Male/female, n = 352	184/168 (52.3/47.7)
Diagnosis, n = 354
Liver disease	222 (62.7)
Irritable bowel syndrome	132 (37.3)

Questionnaires

Fatigue Impact Scale

The FIS is a 40-item instrument assessing functional limitations attributed to fatigue in three domains of daily life: physical functioning (10 items), cognitive functioning (10 items), and psychosocial functioning (20 items) [17]. Table 2 shows these domains and short item names. Complete item expressions can be found in Fisk et al. [17]. Each item is rated on a five-step Likert scale, where 0 = no problems to 4 = extreme problems during the previous month. Ratings are summed to a total score, ranging from 0 to 160, and domain scores range from 0 to 40 for physical and cognitive functioning and 0–80 for the psychosocial functioning. Higher scores indicate greater limitations in functioning. The Swedish version of the FIS has been psychometrically evaluated in patients with multiple sclerosis and in the general population [33].

Table 2. Comparative item sets of FIS versions based upon the FIS item number.

FIS item number and short name^a	Subscale	MFIS	D-FIS	FIS15	FIS6
1. I feel less alert	Cognitive	X	X
2. Isolated from social activities	Psychosocial
3. Reduced my workload	Psychosocial		X	X	X
4. I am more moody	Psychosocial
5. Difficulty paying attention	Cognitive	X
6. I feel like I cannot think clearly	Cognitive	X
7. I work less effectively	Psychosocial
8. Rely on others to help me	Psychosocial
9. Difficulty planning activities	Psychosocial			X
10. More clumsy and uncoordinated	Physical	X		X	X
11. I am more forgetful	Cognitive	X
12. More irritable and angered	Psychosocial
13. Careful about physical activities	Physical	X
14. Less motivated to do physical effort	Physical	X	X
15. Engage in social activities	Psychosocial	X		X
16. Ability to travel outside home	Psychosocial	X		X
17. Trouble maintaining physical effort	Physical	X	X
18. Difficult to make decisions	Cognitive	X	X	X	X
19. Few social contacts outside home	Psychosocial
20. Normal events are stressful	Psychosocial			X
21. Less motivated to do thinking	Cognitive	X
22. Avoid stressful situations	Psychosocial			X
23. Muscles feel weak	Physical	X
24. Increased physical discomfort	Physical	X		X	X
25. Difficulty dealing with anything new	Psychosocial			X
26. Ability for thinking tasks	Cognitive	X	X
27. Ability to meet demands	Psychosocial
28. Ability to provide financial support	Psychosocial			X
29. Engage in sexual activity	Psychosocial
30. Difficult to organize thoughts	Cognitive	X		X	X
31. Ability for physical tasks	Physical	X
32. Worry how I look to other people	Physical			X
33. Less able to deal with emotional issues	Psychosocial
34. Feel slowed down in my thinking	Cognitive	X	X
35. I find it hard to concentrate	Cognitive	X
36. Participating in family activities	Psychosocial			X
37. Limited physical activities	Physical	X	X
38. More or longer periods of rest	Physical	X		X	X
39. Unable to provide emotional support	Psychosocial
40. Minor difficulties get major	Psychosocial

^a All items start with: Because of my fatigue….

Modified Fatigue Impact Scale (MFIS)

The MFIS was created during the development of the Multiple Sclerosis Quality of Life Inventory (MSQLI) [34] by shortening the 40-item FIS. The items of the MFIS were selected by the National Multiple Sclerosis Society for use as a part of the assessment of health-related quality of life in patients with MS [35]. This selection resulted in 21 of the original items (physical functioning (nine items), cognitive functioning (10 items), and psychosocial functioning (two items)) [29,36,37] (Table 2). Rating scales and scoring procedures are the same as for the FIS. For unclear reasons, the response format on the five-step Likert scale was changed in the MFIS-version from MS Society from rating the intensity of the problems (no problem to extreme problem) to rating their frequency (never to almost always). For the purpose of our study, the response format is identical to the original 40 item FIS.

Daily Fatigue Impact Scale (D-FIS)

The D-FIS is an eight-item version of the FIS designed for daily use in clinical practice. Items were selected from the original pool of FIS items by means of Rasch analysis. It comprises three items from the physical domain, four items from the cognitive domain, and one item from the psychosocial domain [18] (Table 2). Rating scales and scoring procedures are the same as for the FIS; however, only a single score is computed.

Rasch analysis

Rasch analysis was used to test the requirements for fundamental measurement [38,39], including stochastic ordering of items (a probabilistic form of Guttman Scaling), local independence of items (conditional upon the trait), unidimensionality [40], ordering of thresholds in polytomous response options [41], and scale invariance across groups (differential item functioning, DIF) which was tested for age, gender, and diagnosis [42]. Full details of the process are given elsewhere [43,44]. Briefly, Item misfit was considered if their χ² value was below the Bonferroni adjusted Chi-Square level. Residual fit values were considered acceptable within the range ±2.5. The local independence assumption for items was tested by the magnitude of residual correlations, whereby a value of 0.2 above the average residual correlation was considered a breach of the assumption [45]. Unidimensionality was tested post hoc by the procedure introduced by Smith, where the lower confidence interval of the number of significant person-specific estimates derived by contrasting two independents sets of items should be below 5% [46].

Finally, a strategy to deal with clusters of locally dependent items was to group them into testlets, thereby absorbing the impact of local dependency, and re-analyze on this basis [47]. Testlets are thus, in practice, just “super items”, where a set of items are added together to make a new polytomous item with the score range of the summated constituent items. Thus, testlets do not affect the total score range of the scale.

Information from these analyses was used to identify alternative item sets that fulfill the requirements of the Rasch model. Reliability was tested with Cronbach’s alpha. RUMM 2030 (http://www.rummlab.com.au/) was used for Rasch analysis. Descriptive statistics was undertaken with SPSS (Statistical Package for Social Services version 22, SPSS Inc., Chicago, IL).

Ethical approval

Ethical approval for data collection was obtained from the Regional Ethical Review Board in Gothenburg. All participants received verbal and written information, and signed an informed consent form before any study-related procedure took place.

Results

A total of 354 patients, (132 with IBS; 222 with liver disease) completed the FIS 40-item questionnaire from which data for MFIS and D-FIS items were derived. Gender, age, and diagnosis of the patient groups are shown in Table 1. Almost three-fifths (57.4%) were under the age of 55, and approximately half of them were female.

Initially, the original FIS data did not fit the Rasch model, extensive local dependency in the item set was observed, and some DIF by age and diagnosis was also found. The scale showed multidimensionality, and misfit across all indicators of fit (Table 3: FIS). However, the scoring categories of all individual items were ordered correctly, and reliability (Cronbach’s alpha) was high, consistent with individual (high stakes) use. The patterns of local dependency mapped to the underlying conceptual structure of the physical, cognitive and psychosocial domains. Consequently, the items were grouped into domain-based testlets and the analysis was re-run. This showed improvement in the indicators of fit with a marginal fall in the level of reliability (Table 3: FIS-3). This solution yielded a unidimensional scale, but some DIF by age and diagnosis remained. The domains were split by age to accommodate the DIF, and this resulted in a further improvement in fit (Table 3: FIS-3 (DIF)). DIF by diagnosis disappeared after this adjustment, suggesting an age-diagnosis interaction. However, the Cognitive and Physical domains were found to be marginally biased in opposite directions for both contextual factors, with low overall differences in mean estimates (Cognitive 0.13 logits; Physical 0.41 logits). After splitting the Physical domain for age, the artificial DIF for age in the Cognitive domain disappeared [48]. Person estimates from the testlet analysis and the analysis with the Physical testlet split for age were compared using the unbiased Cognitive and Psychosocial testlets as anchors. Paired t-test between person estimates showed a non-significant p value (p = 0.62) showing that the effect of DIF canceled out at a test level.

Table 3. Fatigue Impact Scale (FIS): summary of Rasch analysis.

	Item residual		Person residual		Chi square interaction
Analysis	Mean	SD	Mean	SD	Value	df	p	α	t-Test % (LCI)	% at floor
FIS	–0.208	1.678	–0.340	1.920	331.8	200	0.0000	0.94	22.3 (19.9)	11.3
FIS-3	–0.516	2.156	–0.516	1.024	27.2	21	0.1631	0.84	3.5 (0.9)	11.3
FIS-3(dif)	–0.065	1.336	–0.463	0.995	35.9	42	0.7351	0.96	NA	11.3
MFIS	–0.291	0.925	–0.403	1.587	93.6	105	0.7806	0.97	24.7 (22.3)	11.9
MFIS-3	–0.214	1.338	–0.415	0.939	33.8	15	0.0037	0.78	4.7 (2.1)	11.9
MFIS-3(dif)	0.198	0.966	–0.405	0.938	40.5	25	0.0267	0.80	NA	11.9
D-FIS	–0.525	1.157	–0.360	1.072	36.4	40	0.6320	0.85	9.7 (7.3)	17.5
FIS15	–0.525	1.536	–0.408	1.316	67.7	75	0.7140	0.92	7.3 (4.9)	15.2
FIS6	–0.345	0.625	–0.341	1.032	28.3	30	0.5551	0.89	6.5(4.0)	20.0
Ideal values	0.0	1.0	0.0	1.0			>0.05^a	>0.7	(LCI <5.0)

^a Bonferroni adjusted, LCI: lower confidence interval; NA: not applicable.

The 21 items of the MFIS, embedded within the FIS, showed that while fit and reliability were acceptable, high levels of local dependency were evident, and significant levels of multidimensionality, together with some DIF (Table 3: MFIS). The same domain-based testlet solution as above was implemented, with similar results: local dependency disappeared and a strictly unidimensional scale was evident (Table 3: MFIS-3); nevertheless, some DIF remained. The D-FIS had adequate summary fit to the Rasch model; however, significant multidimensionality was observed and half of the items showed local dependency (Table 3: D-FIS). DIF by age and diagnosis was also found. This could not be resolved by a testlet solution.

Given the observed redundancy within the item set implied by the local dependency, and the presence of DIF, an attempt was made to identify an alternative version of the scale which satisfied the full expectations of the Rasch model, including unidimensionality and the absence of DIF. Items were eliminated in an iterative fashion and for example, where pairs of locally dependent items were observed, the worst fitting and/or greatest DIF item was removed. Subsequently, a 15-item scale did satisfy such expectations (Table 3: FIS15), and a six-item scale was also derived for daily clinical use (Table 3: FIS6). Figure 1 shows the graphical representation of the raw score ranges of each scale on the common metric where the floor effect clearly shows for the smaller scales. Note that the % at the floor of each scale increases as the number of items decreases by 6% from the D-FIS to the FIS6. However, irrespective of scale, the majority of those at the floor were those with liver disease. One item (item 1, I feel less alert) had extensive DIF by age and diagnosis, and could not be used, despite its position reflecting the mildest level of fatigue. Figure 2 shows the person-item distribution of the FIS15, where 80% of those at the floor of the scale are those with liver disease. Table 2 shows the item numbers and short names for the various versions of the scales, based upon the ordering of the 40 items in the FIS.

Figure 1. Operational ranges of raw scores of various FIS versions.

Figure 2. Person-item threshold distribution for FIS15.

The results allow for a transformation table (Supplementary Table S1) showing for each ordinal raw score its interval scale Rasch estimates, denoted _RM, of fatigue for the FIS, MFIS, FIS15, and FIS6. These have the same operational range as the raw score. In addition, each scale has another transformation to the same metric as the FIS (40 items), denoted_40M. This allows for using the basic _RM transformation of the FIS as a means of comparing different versions. For example, a raw score of 2 on the FIS6, giving a FIS6_40M value of 51.6, can be seen to be equivalent to a raw score of 18 on the FIS (where its FIS_RM value is also 51.6). The nearest 40_M equivalent score should be taken in such circumstances. For example, a raw score of 11 on the MFIS (with a value of 52.4 on the MFIS_40M) would be equivalent to a score of 19 on the FIS (which as a value of 52.5), see Supplementary Table S1.

Discussion

Fatigue is one of the most disabling symptoms in patients with GI and liver diseases. In these diagnostic groups valid and reliable yet short easy-to-use instruments for assessing fatigue are useful [1–5,8–12,14]. This study examined construct validity, reliability (Cronbach’s alpha), and sufficiency of using a summed score of existing versions of the FIS with 40, 21 or eight items using Rasch analysis and new item sets with 15 and 6 items are suggested for use in clinical research and in clinical settings. The derived MFIS had the same response options as the FIS.

The FIS was heavily locally dependent, with only 11 out of 40 items not residually correlated with at least one other. That is, after the fatigue score is conditioned out, there still remained significant correlations between items, a breach of the local independence assumption required for summative scales. Several items displayed residual correlations with 5–7 other items. These residual correlations were observed to reflect the underlying conceptual structure of the Cognitive, Physical and Psychosocial domains suggesting an association between the apparent high level of multidimensionality and the local response dependence. While local dependency can be accommodated by the testlet approach, it does raise questions about the efficiency of the item set and the potential redundancy therein [17]. Presumably, this was one of the reasons why the MFIS [36] was developed. Unfortunately, the MFIS was also found to be heavily affected by local dependency and resultant multidimensionality, but again this could be resolved by the testlet approach. As such, both of these original scales have considerable redundancy within their respective item sets and at the item level, this would spuriously inflate their reliability. The small magnitude DIF that could be detected was associated between an interaction of age and diagnosis, and once age was accounted for, the diagnosis-related DIF disappeared and therefore should not have a negative impact when the whole scale is used. The DIF for age disappeared at a test level.

The D-FIS was more problematic, showing local response dependency (half of items) and multidimensionality, suggesting that this may not be the most appropriate set of items for a short form of this nature. Thus two further scales (FIS15 and FIS6) were resolved in which the simple summed scale met all the requirements of the Rasch model, including unidimensionality and invariance by age, gender, and diagnosis.

Given that almost all items had ordered thresholds, there was no need for rescoring of items, and thus a straightforward transformation table could be made from the familiar total raw scores of the FIS and MFIS, together with the alternative FIS15 and the FIS6, to both Rasch metric (_RM) with the same range as the raw score, and a common metric based upon the FIS (_40M). After transformation of raw scores to Rasch interval level metric parametric statistical analysis can be used [49,50]. Under these circumstances, the choice of scale version should be led by clinical/epidemiological requirements. In the current cohorts, even the FIS showed an 11% floor effect, whereas it rose to 17% in the D-FIS and 20% for the FIS6. In a clinical sample which experiences considerable fatigue, this may not be an issue as most patients will be well above the floor of these scales. There is also the worrying aspect that the item which represents the entry point into fatigue for the FIS and MFIS (item 1, I feel less alert), and so reduces the floor effect, was found to have considerable bias. Furthermore, score steps at the extremes of the Rasch metric are larger than at the center, indicating that differences in raw scores must be interpreted with caution [51]. This has also been demonstrated in other fatigue scales [52].

That previous modern psychometric evaluations of the scales have almost always indicated lack of fit to either the Rasch model [29] or classical factor analytic approaches (correlated errors in the classical approach), reflect the lack of attention to the issue of local dependency, a shortfall that has been observed with other scales [53,54]. Delivering misfit and multidimensionality under the Rasch model, or bloated specific factors in classical approaches [55], the presence of local dependency has a corrosive effect upon scale interpretation.

The presence of DIF made it necessary to split the items of the FIS and MFIS to obtain the best solution, suggesting that scale interpretation may be subjected to bias under normal circumstances. The FIS15 is free of such bias, and Supplementary Table S1 provides a transformation from this to the other scales and so may be the better choice while still allowing comparability across scale versions. Unfortunately one item (item 1, I feel less alert) which marked the lower end of the scale (so reducing the floor effect) had extensive DIF by age and diagnosis, and could not be used for the versions which satisfied the Rasch model expectations.

There are several limitations to the study. Only the FIS was administered, rather than each of the other variations. Thus, the interpretation of the construct validity of the other versions is based upon their respective item sets drawn from the FIS. The derived MFIS, therefore also had the same response option as FIS and therefore our data may not be adequate for the commonly used version of MFIS. DFIS was not found to give a valid score and we chose to exclude DFIS scores in the transformation table. DIF could only be tested for age, gender, and diagnosis but not for different duration or severity of illness. Also, DIF could exist between different cultural groups but was not tested in this study. For purposes of this study, fatigue is supposed to be measurable and comparable between different groups of patients. The IBS study was performed in patients seeking health care in a secondary/tertiary care clinic, while the majority of IBS patients are managed in primary care, which limits the generalizability to this population. Likewise, the liver cohort consisted of patients with mild liver disease and no patients with severe liver disease were included, so generalizability of the findings to patients with severe liver disease is not possible. Furthermore, as we were dependent upon the available clinical information, we did not use data on disease duration, nor did we use any information about the non-responders.

Given these limitations, and for the frame of reference of these two conditions, having taken care of much of the disturbance to the Rasch model assumptions through a testlet design, all forms of the FIS, other than the original D-FIS, appear adequate, although the newly introduced FIS15 and FIS6 should avoid any problems with bias by age which may affect estimates of the original scales when item sets are not complete.

Conclusions

FIS and MFIS are widely used fatigue scales and despite some shortcomings, these scales can be used to measure fatigue in patients with GI and liver diseases. However, in this paper, we present and recommend alternative 15- and 6-item versions of FIS which can offer valid summed scores with a restricted number of items compared to the 40-item FIS. FIS15 is shorter than FIS with a slightly larger floor effect. FIS6 has a valid sum score and is suitable for daily use. The D-FIS is not valid and should not be used. The transformation table allows raw ordinal scores to be converted to interval equivalent scores for use in parametric statistical analyses, and for comparisons across versions.

Disclosure statement

The authors declare no conflicts of interest.