Psychometric properties of the Insomnia Catastrophizing Scale (ICS) in a large community sample

ABSTRACT

The purpose was to investigate the psychometric properties of the Insomnia Catastrophizing Scale (ICS) including factorial validity and internal consistency as well as discriminative and convergent validity. Associations with sleep parameters and daytime impairment are also examined. Drawn from a randomly selected sample of the general population, 1615 participants completed a survey on insomnia-related nighttime and daytime symptoms, health outcomes and psychological processes, including the ICS. A one-factor solution was supported for both the nighttime catastrophizing (11 items) and daytime catastrophizing (6 items) subscales. Both subscales displayed high internal consistencies (α > 0.90) and accounted for 59.1–70.1% of the variance. The insomnia disorder group had significantly higher scores than participants without insomnia on the two subscales and on the individual items. Cutoffs were established for both subscales with acceptable sensitivity and specificity. Both subscales displayed adequate convergent validity with measures indexing worry, cognitive pre-sleep arousal and anxiety. The two subscales were also significantly associated with nighttime and daytime insomnia symptoms. The ICS is a reliable and valid scale for the assessment of insomnia-related catastrophizing. Future research is needed to examine the test-retest reliability and treatment sensitivity of the ICS.

KEYWORDS:

• Insomnia
• sleep
• catastrophizing
• scale

Insomnia disorder is characterised by difficulties with sleep onset, sleep maintenance, or early-morning awakenings (American Academy of Sleep Medicines, [AASM] 2014; American Psychiatric Association [APA], 2013). These nocturnal symptoms are associated with clinically significant distress or impairment of daytime functioning, including fatigue, decreased energy, mood disturbances, and reduced cognitive functions. A diagnosis of insomnia disorder requires sleep difficulties that are present for three nights or more per week and lasts for more than 3 months (AASM, 2014; APA, 2013). The prevalence rate of insomnia disorder is approximately 10% in the population but may range between 3.9% and 22.1%, depending on the diagnostic systems used (Roth et al., 2011). Insomnia disorder is often associated with medical and mental disorders as well as with significant direct and indirect costs (Morin et al., 2015).

There has been a growing interest in the importance of intrusive and worrisome thinking in the maintenance of insomnia (e.g. Gross & Borkovec, 1982; Espie, 2002; Harvey, 2002; Lundh & Broman, 2000; Morin, 1993). According to several models of insomnia, intrusive thoughts prior to sleep and during nightly awakenings are viewed as exacerbating the sleep difficulties associated with the condition (Espie, 2002; Harvey, 2002; Lundh & Broman, 2000; Morin, 1993). Two specific thought processes that have received some attention in the insomnia literature are worry and rumination. The term worry refers to ‘a chain of thoughts and images, negatively affect-laden and relatively uncontrollable’ (p. 10; Borkovec, Robinson, Pruzinsky, & DePree, 1983), and the term rumination has been used to refer to the repetitive focusing on the ‘causes, meanings and consequences’ of one’s feelings and symptoms (Nolen-Hoeksema, 1991, p. 567). A well-documented finding is that patients with insomnia complain that they cannot get to sleep because of unpleasant intrusive thoughts and excessive and uncontrollable worry during the pre-sleep period (Gross & Borkovec, 1982; Espie, Brooks, & Lindsay, 1989; Harvey, 2000; Lichstein & Rosenthal, 1980). Research on rumination in insomnia is starting to accrue (Carney, Edinger, Meyer, Lindman, & Istre, 2006; Carney, Harris, Moss, & Edinger, 2010).

A related construct to worry and rumination, catastrophizing, involves ‘dwelling on the worst possible outcomes of any situation in which there is a possibility for an unpleasant outcome. The person overemphasises the probability of this catastrophic outcome and usually exaggerates the possible consequences of its occurrence’ (p. 33, Beck, Emery, & Greenberg, 1985). At the core of their definition of catastrophizing was the concept of an irrationally negative forecast of future events. Catastrophizing often involves ‘the worrier persistently posing internal, automatic questions of the ‘what if?’ kind’ (p. 83, Startup & Davey, 2001). In the context of insomnia, the clinical treatment literature (e.g. Morin, 1993; Perlis et al., 2000) highlights the tendency for patients with insomnia to catastrophize the consequences of sleep loss and the negative impact on daytime functioning. In addition, Espie’s psychobiological inhibition model (2002) identifies catastrophizing as a meaningful contributor to insomnia. More specifically, Espie proposes that catastrophizing is a cognitive style that might have a negative impact on sleep homeostasis, circadian timing and sleep misperception. Although not explicitly emphasised in the cognitive model of insomnia (Harvey, 2002), catastrophizing might be viewed as one form of thought process that is involved in maintaining insomnia. A common theme in the abovementioned conceptualizations of insomnia is that catastrophizing is believed to have an impact upon poor sleep and daytime impairment via the increased somatic arousal and cognitive activity that catastrophizing triggers.

There is some preliminary evidence that suggests that catastrophizing might maintain insomnia. In one study (Harvey & Greenall, 2003), a catastrophizing interview (i.e. a questioning process to capture likelihood, anxiety, and discomfort concerning catastrophizing) was used among individuals with insomnia disorder and controls. Relative to good sleepers, catastrophic worry about the consequences of not sleeping was more common and catastrophes were perceived as more likely in those with insomnia disorder. Also, catastrophizing about the consequences of not sleeping resulted in elevated anxiety and discomfort in those with insomnia disorder, but not in good sleepers. In a second investigation, poor and good sleepers were asked to, across three interview steps, catastrophize about two topics (worries about sleep and a current personal worry) and to iterate the positive aspects of a hypothetical topic (Barclay & Gregory, 2010). Relative to good sleepers, poor sleepers generated a higher mean number of steps to the catastrophizing interviews.

Given the limited evidence on thought processes in insomnia, more research is warranted to inform theory and clinical practice. Although it is yet too early to be decisive about the inter-relationship between worry, rumination and catastrophizing in the context of insomnia, there might be certain merits in exploring the role of catastrophizing for insomnia. Although catastrophizing has been shown to act as an important predictor in other areas of health psychology (Flink, Boersma, & Linton, 2013), mental disorders in general (Moore, Adams, Ellis, Thibault, & Sullivan, 2018) and in obsessive-compulsive disorder (Muller & Roberts, 2005), one disadvantage is that catastrophizing has received relatively little attention in the context of insomnia. Another benefit might be that catastrophizing, relative to worry and rumination, is a more specific, extreme thought process, i.e. it captures a distinct thought phenomenon, as well as being more negatively affect-laden and uncontrollable. Another merit in examining the link between catastrophizing and insomnia might be that previous research within the anxiety literature has shown that the tendency to catastrophize might exacerbate the adverse effects of pathological worry (Kendall & Hollon, 1989; Ingram & Kendall, 1987). In all, a deepened focus on catastrophizing would provide insight into how catastrophic thoughts operate and inter-relate with other processes in the context of insomnia, which might have clinical benefits, such as the possibility to address catastrophizing in clinical practice.

Based on cognitively-oriented models of insomnia, it thus seems likely that catastrophizing might be involved in the maintenance of insomnia. One important obstacle to progress has been a reliable and valid assessment of insomnia-related catastrophizing. Recently, a self-report scale, the Catastrophic Thoughts about Insomnia Scale, was developed to index catastrophizing within the context of insomnia (CTIS; Tan, Hadjistavropoulos, & MacNab, 2017). The CTIS was based on a pain catastrophizing measure and intended to index three forms of catastrophizing, i.e. rumination, magnification and helplessness. In a student sample, the CTIS and its subscales had excellent internal consistency and strong associations with insomnia symptoms and unhelpful beliefs about sleep. The CTIS is a promising instrument for the detection of catastrophizing in insomnia. However, based on different theoretical underpinnings, we suggest another way to tap into insomnia catastrophizing. First, since the cognitive model of insomnia (Harvey, 2002) propose that insomnia and the psychological processes that maintain the condition should be viewed as a 24-h condition, we developed and validated an insomnia-specific catastrophizing scale that assesses nighttime and daytime catastrophizing separately. Developing two subscales of insomnia catastrophizing would, at least theoretically, enable the identification of unique characteristics during the night as well as the day. Second, we based our scale development on the definition of catastrophizing as suggested by Beck et al. (1985) and made some minor changes (for details see the Methods section). Following the revised definition, at least one of the forms of catastrophizing (i.e. rumination) assessed in the CTIS does not fit theoretically due to that rumination is commonly viewed as a thought process that content-wise consists of present and past experiences (Smith & Alloy, 2009) and not catastrophic interpretation of future events.

The aims of the study were therefore to develop a new scale as an index of insomnia catastrophizing and examine its factorial validity, internal consistency, discriminant validity, convergent validity and associations with sleep parameters and daytime impairment.

Methods

Participants

This research is part of the Prospective Investigation on Psychological Processes for Insomnia (PIPPI) study, which was approved by the Regional Ethics Board in Uppsala, Sweden. A random sample of 5,000 residents from two counties in Sweden (Örebro and Värmland), 18–70 years old, were sent surveys at three time points. The first survey at wave I was sent out in September 2008. The random sample was obtained from the national register in which all residents are listed. Of the total sample, 58 (1.2%) were not eligible (incorrect address: n = 38, participation refusal: n = 20). Of the 4942 eligible residents, 2333 participants (47.1%) returned the survey. Comparisons with register data showed that our sample was representative of the Swedish population regarding age, gender, relationship status, occupational status, educational level, and reported sleep disturbance (Jansson-Fröjmark et al., 2012). Attrition analyses showed no differences for gender, sleep disturbance or insomnia severity although non-responders were more likely to be younger than responders (Jansson-Fröjmark et al., 2012).

The wave I respondents were sent two additional surveys, one in March 2009 (6-month follow-up; wave II) and one in March 2010 (18-month follow-up; wave III). The current study uses data from wave III since that was the only time that the ICS was included in the survey. In total, 1795 of the 2333 wave I participants (76.9%) returned the survey at wave III. There were no statistical differences between the wave I and the wave III responders in terms of socio-demographic data or any other measures used in this paper.

The inclusion criterion that was used for this paper required that the participants did not fulfil criteria for a sleep disorder other than insomnia. The SLEEP-50 was used to assess six DSM-IV-TR sleep disorders: sleep apnea, narcolepsy, restless legs/periodic limb movement disorder, circadian rhythm disorder, and sleep walking (Spoormaker, Verbeek, van Den Bout, & Klip, 2005). The instrument has high internal consistency, test-retest correlation ranging between .65 and .89, and a factor structure that matches the DSM-IV-TR sleep disorders. The sensitivity and specificity scores have been found to be reasonable for the sleep disorders (sensitivity: 0.67–1.00; specificity: 0.69–1.00). The agreement between clinical diagnoses and classification derived from the SLEEP-50 is substantial (kappa = 0.77) (Landis & Koch, 1977). The participants were asked to rate to what extent the items have been applicable during the past month (1 = not at all, 4 = very much). Of the 1795 study participants at wave III, 5.6% scored above the SLEEP-50 cutoffs for sleep apnea, 0.5% for narcolepsy, 6.5% for restless legs/periodic limb movement disorder, 2.5% for circadian rhythm disorder, and 0.2% for sleep walking. In all, 10.0% (n = 180) scored above the SLEEP-50 cutoffs for at least one of the six sleep disorders and was therefore excluded. Thus, 1615 participants were included in this study.

Of the 1615 study participants, the mean age was 51.2 years (SD = 14.0), 56.5% were women, and 93.7% were born in Sweden. As for marital status, 12.0% reported being single, 82.0% being cohabitant or married or having a partner, 4.1% being divorced, and 2.0% being widowed. Regarding vocational status, 66.9% were employed (full or part-time) or students and 33.1% were unemployed, on sick leave, on pension or other status. Concerning educational level, 24.1% had compulsory school as their highest level of education, 42.5% high school, and 33.4% college or university. Compared with descriptive statistics from public register data reflecting the population in Sweden (e.g. Statistics Sweden and Swedish National Institute of Public Health), the current sample was relatively representative on several demographic parameters. The mean age for all Swedish residents, including those below 18 years and above 70 years, is 41 years. For the population of 16–74 year olds in Sweden, 49.5% are women. According to register data, approximately 87% report being born in Sweden. Of all Swedish residents, 34.1% report being married, 51.2% unmarried, 9.4% divorced, and 5.3% widowed. In the population of 16–74 year-old residents in Sweden, 82% are employed and level of education is as follows: 22.6% compulsory school, 44.5% senior high school, and 32.9% college or university.

Procedure

The survey was mailed at wave III to the wave I responders. It was accompanied by an introductory letter and invitation to participate as well as a pre-paid return envelope. If a response was not received within two weeks a reminder was mailed. If an additional two weeks elapsed without a response a new survey was sent. To increase the response rate, a number of steps were taken in line with a Cochrane review (Edwards, Clarke, DiGuiseppi, Pratap, & Wentz et al., 2007). Specifically, we sent a pre-notification letter, a small incentive, an information letter describing the project’s aim, relevance, and an assurance of confidentiality, a pre-paid return envelope and reminders to non-responders. Also, we used a project webpage, a multicoloured and user-friendly survey, closed questions, and placed relevant and easy questions first.

Measures

Demographic variables

All study variables, including the demographic parameters, were measured by self-report. The following demographic parameters were assessed: age, gender, civil status, level of education, vocational status and place of birth.

Nighttime symptoms

To assess sleep disturbance, the participants were asked to complete the following categorical questions based on the past month: sleep onset latency (SOL; <15 min, 16–30 min, 31–60 min, >60 min), wake time after sleep onset (WASO; same alternatives as for SOL), early morning awakening (EMA; same alternatives as for SOL), total sleep time (TST; < 4 h, 4–5 h, 5–6 h, 6–7 h, 7–8 h, 8–9 h, 9–10 h, >10 h), sleep restoration [completely (1), a lot (2), somewhat (3), a little (4), not at all (5)], and sleep quality [very good (1), quite good (2), neither good nor poor (3), quite poor (4), very poor (5)]. To determine sleep disturbance, the participants were asked to complete the following questions based on the past month: sleep disturbance (yes or no; if no: continue to daytime impairment section), and frequency of sleep disturbance (<1 night per week, 1–2 nights per week, 3–5 nights per week, every night).

Daytime symptoms

The participants were instructed to report on the degree of sleep-related impairment during the past month: fatigue/malaise, impairment in attention, concentration, or memory, social dysfunction, vocational dysfunction, mood disturbance, irritability, daytime sleepiness, reduction in motivation, energy, or initiative, proneness for errors or accidents at work or while driving, tension headaches, gastrointestinal symptoms, and concerns or worries about sleep (Edinger et al., 2004). The response alternatives for these indications of daytime impairment were: not at all (1), somewhat (2), quite much (3) and a lot (4). The response alternatives for the two functional domains (i.e. social and vocational dysfunction) were: no negative consequences (1), small negative consequences (2), marked negative consequences (3), large negative consequences (4) and very large negative consequences (5). To form a composite score for daytime impairment, all the impairment items were summed.

Catastrophizing

When beginning the task of constructing a new insomnia-related catastrophizing scale, we made several changes to the definition of catastrophizing as formulated by Beck et al. (1985). We avoid the term ‘dwelling’ for two reasons; first, it implies worry or rumination; second, research within affective neuroscience has shown the importance of fast bottom-up, automatic interpretations in the catastrophizing process (Ledoux, 1998). We also decided to delete ‘catastrophizing outcome’ as part of our definition as this is using the term we are trying to define as part of the definition. On a more technical note, we also deleted the last sentence of the original definition since it is repetitive. To conclude, the current work assumed a definition of catastrophizing that reads: ‘catastrophizing involves appraisal of worst possible outcomes of any situation in which there is possibility for an unpleasant outcome’. We also want to highlight that our definition to a high degree resembles current definitions of catastrophizing within the pain literature (Quartana, Campbell, & Edwards, 2009).

Self-report items were first generated individually by the authors. The authors used extant catastrophizing scales, such as from the pain and insomnia areas [e.g. the Pain Catastrophizing Scale (Sullivan, Bishop, & Pivik, 1995) and the Safety Behaviors and Catastrophizing Scale (MacDonald, Linton, & Jansson-Fröjmark, 2008)], themes and items from a previous study on catastrophizing in insomnia (Harvey & Greenall, 2003), and experience from clinical work to generate items. This was followed by a group-wise brainstorming of items and then constructing an extensive, combined item pool, after which several items were deleted due to redundancy or overlap. Guiding principles during this phase were to keep items that were in accordance with the working definition of catastrophizing and items that tapped into both nighttime and daytime catastrophizing. Following this, several colleagues at Örebro University were asked to provide feedback on the items, for example concerning language issues and content validity. Also, 17 patients from a clinical trial were asked to provide feedback on the items (Jansson-Fröjmark et al., 2012). Based on their own experience, the patients were asked to reflect upon the extant items and brainstorm on themes or items that they thought were missing. This work resulted in the construction of the Insomnia Catastrophizing Scale (ICS) consisting of 20 items. The instruction for the nighttime items was: ‘Everyone has a night of poor sleep sometimes. During such nights you might think about your poor sleep and its consequences. Please circle the number that best describes how often you notice these thoughts’. The instruction for the daytime items was: ‘Everyone has a night of poor sleep sometimes. During the day after you might think about your poor sleep and its consequences. Please circle the number that best describes how often you notice these thoughts’. The response alternatives for each of the items were in terms of occurrence of catastrophic thoughts (0–5; 0 = never, 5 = always).

Preliminary findings from the current study have previously been presented as a poster (Jansson-Fröjmark, Harvey, & Flink, 2012). The ICS has also been translated to Italian and shown to display acceptable psychometric properties, including support for two distinct catastrophizing subscales (i.e. nighttime and daytime catastrophizing) and significant associations with insomnia severity (Ballesio et al., 2018).

Sleep-related worry

The Anxiety and Preoccupation about Sleep Questionnaire was used to assess sleep-related worry (APSQ; Jansson-Fröjmark, Harvey, Lundh, Norell-Clarke, & Linton, 2011; Tang & Harvey, 2004). The response alternatives for each of the 10 items were 1 (‘strongly disagree’) to 5 (strongly agree). The score range was thus 10–50. Based on the current sample, the internal consistency for the APSQ was high a α = 0.93 (Kline, 1993).

Cognitive pre-sleep arousal

The cognitive subscale from the Pre-Sleep Arousal Scale was employed to determine cognitive pre-sleep arousal (PSAS-C) (Nicassio, Mendlowitz, Fussell, & Petras, 1985). Based on previous psychometric evaluation of the PSAS-C, the subscale was limited to five items instead of the original eight items (Jansson-Fröjmark & Norell-Clarke, 2012). The response alternatives were 1 (‘not at all’) to 5 (‘extremely’) with a score range of 8–40. In the current sample, the internal consistency for the cognitive subscale was high at α = 0.92 (Kline, 1993).

Anxiety

The anxiety subscale of the Hospital Anxiety and Depression Scale was used to assess anxiety (HADS-A; Zigmond & Snaith, 1983). The subscale consists of seven anxiety questions in which the severity is rated on 4-point scales (score range 0–21). Based on the current sample, the internal consistency for the anxiety subscale was high (α = 0.85) according to Kline’s criteria (1993).

Sleep groups

The participants were classified in one of two groups (insomnia disorder vs not insomnia disorder) according to their sleep patterns, daytime impairment, and evidence of sleep disorders other than insomnia. The classification used an algorithm based on a combination of insomnia diagnostic criteria from Research Diagnostic Criteria for insomnia (Edinger et al., 2004), established quantitative criteria for insomnia (Lichstein, Durrence, Taylor, Bush, & Riedel, 2003; Pillai, Roth, & Drake, 2016), and screening for sleep disorders other than insomnia (Spoormaker et al., 2005). Note that individuals who scored above cutoffs on the SLEEP-50 concerning six primary sleep disorders (i.e. sleep apnea, narcolepsy, restless legs, periodic limb movement disorder, circadian rhythm disorder and sleep walking) were excluded from this study.

Insomnia disorder: To belong to this group, three diagnostic criteria had to be met. (1) The participant had to affirm a sleep disturbance during the past month. (2) The individual had to report initial, middle or late insomnia (31 min or more per night; Lichstein et al., 2003) or non-restorative sleep (‘a little’ or less) or poor sleep quality (‘quite poor’ or less). This sleep pattern had to be present for at least three nights per week. (3) The participant had to report daytime impairment (for symptoms: ‘quite much’ or more, for function: ‘marked negative consequences’ or more).

Statistical analysis

To reveal any latent variables within the ICS that cause the manifest variables to co-vary, exploratory factor analysis was used in line with recommendations (Costello & Osborne, 2005). To ensure that the characteristics of the data set were suitable for the factor analysis to be conducted, the Kaiser-Meyer-Olkin measure of sampling adequacy (KMO) and the Bartlett Test of Sphericity (BTS) were conducted on the data. A maximum likelihood factor extraction procedure with oblique rotation (direct oblimin) was employed since this approach is particularly useful in extracting psychologically meaningful factors and because of the possibility that the extracted factors may be correlated (Costello & Osborne, 2005). The parallel analysis (Horn, 1965; O’Connor, 2000) was a basis for decisions of how many factors that should to be retained for rotation. The minimum loading of an item was determined at 0.32, a recommended threshold for the minimum loading of an item (Tabachnick & Fidell, 2001). To investigate internal consistency, Cronbach’s alpha was used, and 0.70 was considered as the minimum acceptable criterion of instrument internal reliability (Kline, 1993). The discriminant validity was examined with analysis of variance in which group (insomnia disorder vs not insomnia disorder) was used as the fixed factor and the total ICS subscales as well as the ICS items as the dependent variables. Assumptions for ANOVA were checked before executing the analyses. Receiver operating characteristics analyses were also used to examine the discriminate validity of the two subscales. The analyses concerning convergent validity and association between (a) the ICS subscales and (b) sleep parameters and daytime impairment employed a correlative approach [Spearman’s Rho (Colton, 1974), Eta, and Contingency Coefficient]. In the latter analyses, the sleep parameters were dichotomised as follows: sleep onset latency, wake after sleep onset, and early morning awakening into 30 min or less versus more than 30 min (Lichstein et al., 2003) and total sleep time into more than 6 h versus 6 h or less (Vgontzas et al., 2010). The magnitude of the correlations using Spearman’s Rho was considered using the guidelines from Colton (1974) where 0.00–0.25 = little or no relationship, 0.25–0.50 = a weak to fair relationship, 0.50–0.75 moderate to good relationship and 0.76 and above considered good to excellent.

Results

Selection of items for the ICS

The 20 ICS items were analyzed via exploratory factor analysis. The 20 nighttime items were first separately assessed in terms of their communalities (low communality: below 0.40). One of the twelve nighttime items displayed a low communality (‘I can’t stand being awake all night’) and was therefore removed. Also, two of the eight daytime items exhibited low communalities (‘There is nothing I can do to feel better today’ and ‘I won’t be able to cope all day’). In total, 17 items were kept for further analyses. Due to our intention to construct a scale determining nighttime and daytime insomnia catastrophizing and the similarity of a few items across the nighttime and daytime domains (‘My poor sleep will have serious consequences’ versus ‘My poor sleep will have serious consequences today’ and ‘My ability to function will be seriously affected’ versus ‘My daily activities will be seriously affected’), the two subscales were analysed separately.

Factorial validity, internal consistency and inter-relationship

First, we analysed the eleven nighttime catastrophising items [hereafter labelled Insomnia Catastrophizing Scale—Nighttime (ICS-N]. Preparatory analyses were first executed to ensure that the data distribution satisfied the psychometric criteria for a factor analysis. The analyses showed that the KMO yielded an index of 0.93 and the BTS was significant (χ² _{(df = 55)} = 5549.91, p < 0.001). Based on these findings, a factor analysis was viewed as appropriate. The parallel analysis suggested a one-factor solution for the ICS-N, accounting for 59.1% of the variance. The factor loadings, communalities and corrected item-total correlations are displayed in Table 1 for the one-factor solution. The 11 ICS-N items showed strong primary loadings (0.65–0.84), moderate to strong inter-correlations (0.34–0.76), and high internal consistency (α = 0.92). A two-factor solution was also evaluated to examine whether manually setting the factors to retain would improve the factor solution. The two-factor solution, accounting for 67.1% of the variance, consisted of one factor with seven items (numbers 1, 3, 5, 7–10; 59.1% unique variance, Eigenvalue 6.50, factor loadings 0.46–0.83, inter-item correlations 0.38–0.76, α = 0.90) and a second factor with four items (numbers 2, 4, 6, 11; 8.0% unique variance, Eigenvalue 0.88, factor loadings 0.45–0.92, inter-item correlations 0.46–0.66, α = 0.84). Several disadvantages with the two-factor solution was noted (i.e. parallel analysis indicating one factor as well as difficulty interpreting the content difference across the two factors).

Table 1. Exploratory factor analysis of the Insomnia Catastrophizing Scale—Nighttime (ICS-N).

		Factor loading	Communalities	Corrected item-total correlations
1	My poor sleep will always continue	0.72	0.52	0.70
2	There is nothing I can do to get to sleep	0.67	0.45	0.66
3	My physical health will be negatively affected	0.82	0.68	0.78
4	I will get far too little sleep tonight	0.70	0.49	0.69
5	My poor sleep will have serious consequences	0.84	0.71	0.79
6	My thoughts will race all night	0.76	0.57	0.75
7	My poor sleep will get worse	0.77	0.60	0.73
8	My physical appearance will be negatively affected	0.74	0.54	0.70
9	I will lose control over my ability to sleep	0.77	0.59	0.73
10	My poor sleep will result in a nervous breakdown	0.65	0.43	0.60
11	My ability to function will be seriously affected (e.g. work, physical exercise, housework)	0.71	0.50	0.69

Goodness-of-fit test: χ² = 650.92, p < 0.001.

Second, we investigated the six daytime catastrophizing items [hereafter labelled Insomnia Catastrophizing Scale—Daytime (ICS-D]. Preparatory analyses showed that the KMO yielded an index of 0.88 and the BTS was significant (χ² _{(df = 15)} = 3091.33, p < 0.001). A factor analysis was thus viewed as appropriate. The parallel analysis indicated a one-factor solution for the daytime items, accounting for 70.1% of the variance. A two-factor solution was also evaluated to examine whether manually setting the factors to retain would improve the factor solution. The two-factor solution was discarded because of the following reasons: the second factor contained only one item (‘I will feel worse and worse’), the parallel analysis indicated a one-factor solution and the Eigenvalue for the second factor was at 0.52. The factor loadings, communalities and corrected item-total correlations for the one-factor solution are displayed at Table 2. The six ICS-D items showed strong primary loadings (0.74–0.84), and the correlations between the items ranged from 0.55 to 0.73, indicating good correlations. The internal consistency of the ICS-D was α = 0.91.

Table 2. Exploratory factor analysis of the Insomnia Catastrophizing Scale—Daytime (ICS-D).

		Factor loading	Communalities	Corrected item-total correlations
1	I will feel awful all day	0.80	0.63	0.75
2	My poor sleep will have serious consequences today	0.84	0.70	0.80
3	I will feel worse and worse	0.74	0.55	0.71
4	I will function poorly all day	0.83	0.68	0.78
5	My daily activities will be seriously affected	0.84	0.70	0.79
6	I will lose control over my ability to function (e.g. work, physical exercise, housework)	0.76	0.58	0.73

Goodness-of-fit test: χ² = 163.42, p < 0.001.

The two ICS subscales were significantly associated with age (ρ: between −0.06 and −0.12, p < 0.01), gender (η: between 0.06 and 0.08, p < 0.05), and place of birth (η: between 0.13 and 0.14, p < 0.01), but not with civil status, level of education, and vocational status. In all, higher ICS scores were associated with younger age, being a woman, and not being born in Sweden.

Discriminative validity

To investigate the discriminant validity of the ICS, analysis of variance and receiver operating characteristics were used. In the two forms of analyses, the 218 participants with insomnia disorder were compared with the 1397 individuals without insomnia disorder. As can be seen at Table 3, analysis of variance showed that the ICS-N subscale, the ICS-D subscale and the 17 ICS items discriminated the two groups. Across all analyses, the insomnia disorder group had higher scores than the other group on the two subscales and on the individual items (p < 0.001 in all instances). The between-group effect sizes for the ICS-N was 1.47 (range for the 11 ICS-N items: d = 0.68–1.23) and for the ICS-D 1.23 (range for the six ICS-D items: d = 0.76–1.08).

Table 3. Discriminant validity of the Insomnia Catastrophizing Scale at Subscale- and Item-Level.

	Insomnia disorder
	Yes (n = 218)	No (n = 1397)
	M (SD)	M (SD)	t	d
ICS-N	14.9 (10.9)	3.0 (5.4)	15.82**	1.47
Item 1 (N)	1.6 (1.5)	0.2 (0.7)	12.55**	1.23
Item 2 (N)	1.7 (1.5)	0.4 (1.0)	12.37**	1.02
Item 3 (N)	1.4 (1.4)	0.2 (0.7)	12.42**	1.14
Item 4 (N)	2.1 (1.5)	0.6 (1.0)	14.28**	1.22
Item 5 (N)	1.1 (1.3)	0.2 (0.6)	10.06**	1.02
Item 6 (N)	1.6 (1.4)	0.3 (0.8)	13.37**	1.13
Item 7 (N)	1.0 (1.3)	0.1 (0.5)	10.11**	0.98
Item 8 (N)	1.2 (1.5)	0.2 (0.7)	9.68**	0.96
Item 9 (N)	1.0 (1.4)	0.1 (0.4)	9.43**	0.96
Item 10 (N)	0.6 (1.0)	0.1 (0.4)	7.29**	0.68
Item 11 (N)	1.6 (1.4)	0.4 (0.8)	12.35**	1.07
ICS-D	6.4 (5.9)	1.5 (3.2)	11.99**	1.08
Item 1 (D)	1.3 (1.3)	0.4 (0.8)	9.93**	0.82
Item 2 (D)	0.9 (1.2)	0.2 (0.6)	8.45**	0.77
Item 3 (D)	0.8 (1.2)	0.1 (0.5)	8.50**	0.83
Item 4 (D)	1.4 (1.4)	0.3 (0.7)	11.38**	1.01
Item 5 (D)	1.3 (1.2)	0.3 (0.7)	11.99**	0.98
Item 6 (D)	0.8 (1.2)	0.1 (0.5)	8.50**	0.76

ICS-D = Insomnia Catastrophizing Scale—daytime, ICS-N = Insomnia Catastrophizing Scale—nighttime. ** p < 0.001.

Receiver operating characteristics analyses were also performed to examine the discriminate validity of the ICS-N and the ICS-D. The two ICS subscales were tested separately as test variables and insomnia disorder (yes/no) as the state variable. On the ICS-N and the ICS-D, the area under the curve (AUC) values can be viewed as high in accuracy. The optimal sensitivity-specificity combination for the ICS-N was set at 5 points, resulting in 79.7% sensitivity and 77.8% specificity (AUC = 0.88, CI 0.79–0.96, SE = 0.01, p < 0.001). The optimal combination for the ICS-D was located at 2 points, leading to 76.2% sensitivity and 74.5% specificity (AUC = 0.80, CI 0.71–0.89, SE = 0.02, p < 0.001). Note that the confidence interval of the AUC-values did not include 0.5, which indicates that the analysis could discriminate between participants with and without insomnia disorder better than chance.

Convergent validity

The correlations between the two ICS subscales and three related constructs, i.e. sleep-related worry (APSQ), cognitive pre-sleep arousal (PSAS-C), and anxiety (HADS-A), are displayed at Table 4. As depicted at the table, the two ICS subscales were significantly related at a moderate to good level to the APSQ (ρ = 0.70–.75), the PSAS-C (ρ = 0.48–0.56), and the HADS-A (ρ = 0.50–0.52).

Table 4. Correlations between the two ICS subscales, related constructs, sleep parameters, and daytime impairment.

	ICS-N	ICS-D	APSQ	PSAS-C	HADS-A	SOL	WASO	EMA	TST
ICS-N
ICS-D	0.75**
APSQ	0.73**	0.70**
PSAS-C	0.56**	0.48**	0.52**
HADS-A	0.52**	0.50**	0.52**	0.58**
SOL ^a	0.39**	0.26**	0.38**	0.32**	0.19**
WASO ^a	0.29**	0.20**	0.27**	0.29**	0.21**	0.34**
EMA ^a	0.27**	0.23**	0.28**	0.27**	0.24**	0.19**	0.33**
TST ^b	0.31**	0.21**	0.30**	0.22**	0.18**	0.28**	0.34**	0.28**
DI	0.53**	0.60**	0.60**	0.45**	0.56**	0.20**	0.16**	0.21**	0.18**

Spearman’s rho, Eta, and Contingency Coefficient were used as correlation statistics. APSQ = Anxiety and Preoccupation about Sleep Questionnaire, DI = daytime impairment, HADS-A = Hospital Anxiety and Depression Scale—anxiety, ICS-N = Insomnia Catastrophizing Scale—nighttime, ICS-N = Insomnia Catastrophizing Scale—daytime, PSAS-C = Pre-Sleep Arousal Scale—cognitive subscale, SOL = sleep onset latency, WASO = wake after sleep onset, EMA = early morning awakening, TST = total sleep time. ^a 30 min or less versus 30 min or more. ^b More than 6 h versus 6 h or less. ** p < 0.01.

Associations between the ICS subscales with sleep parameters and daytime impairment

The association between the two ICS subscales with sleep parameters and daytime impairment was investigated. First, the two subscales were correlated (Eta) with four categorical sleep parameters [sleep onset latency, wake after sleep onset, and early morning awakening (30 min or less versus more than 30 min); total sleep time (more than 6 h versus 6 h or less)]. As is displayed in Table 4, the two subscales were moderately associated with sleep onset latency (η = 0.26–0.39), wake after sleep onset (η = 0.20–0.29), early morning awakening (η = 0.23–0.27), and total sleep time (η = 0.21–0.31). Second, the two subscales were also correlated (Spearman’s rho) with daytime impairment (total score of the 12 daytime impairment items). In the same table, it is possible to verify that the two subscales were correlated with daytime impairment at a moderate to good level (ρ = 0.53–0.60).

Discussion

The first aim of this investigation was to examine the factorial validity and the internal consistency of the ICS. The analyses supported a one-factor solution for both the nighttime catastrophizing (11 items) and daytime catastrophizing (six items) subscales. The ICS items displayed strong primary loadings and moderate-to-strong inter-correlations. Both subscales displayed high internal consistencies and accounted for approximately 59–70% of the variance.

The second aim was to explore the discriminant validity of the ICS. The discriminant validity was supported in that participants with insomnia disorder had significantly higher scores than participants without insomnia on the two subscales and on the individual items. The between-group effect sizes for the two subscales were large. Note however that a few of the ICS items displayed lower between-group effect sizes (i.e. item 10 in the ICS-N, item 2 in the ICS-D, and item 6 in the ICS-D). Cutoffs were established for both subscales with acceptable sensitivity and specificity, thus showing that the subscales are able to differentiate individuals with insomnia disorder and those without with acceptable accuracy. The discriminant validity was also supported in that receiver operating characteristics analyses showed cutoffs for the two subscales with acceptable sensitivity (76.2–79.7%) and specificity (74.5–77.8%).

The third aim was to investigate the convergent validity of the ICS. Both ICS subscales were significantly associated with three measures that intend to assess anxiety-related constructs, namely insomnia-related worry, cognitive pre-sleep arousal, and general anxiety symptomatology. The results also showed that, of the three anxiety-related constructs, the two subscales were most closely related to insomnia-related worry, an expected finding given that both catastrophizing and worry are often thought off as repetitive thought processes. The results also demonstrated that the nighttime catastrophising subscale, relative to daytime catastrophizing, was slightly more strongly related to the three convergent measures. The latter finding might possibly be explained by that nighttime catastrophizing is likely to be more fearsome given the threat value of facing recurrent or chronic sleep disturbance, thus triggering higher arousal and anxiety levels (Harvey, 2002).

The fourth aim was to examine the association between the ICS with sleep parameters and daytime impairment. The findings showed that the two ICS subscales were significantly associated with sleep onset latency, wake after sleep onset, early morning awakening, and total sleep time. It is noteworthy that the three first nighttime symptoms were categorised as 30 min or less versus more than 30 min since previous research has shown that 31 min per night is a valid cutoff, discriminating individuals with insomnia disorder with normal sleepers (Lichstein et al., 2003). The results also demonstrated that the nighttime catastrophizing subscale, relative to daytime catastrophizing, was slightly more strongly related to the four nighttime symptoms. It is important to underscore that the stronger associations between the ICS-N and nighttime symptoms could be a result of time (i.e. spending more time awake in the bed at night gives you more time to think). Another related finding was that the ICS subscales were more strongly associated with worry, arousal and anxiety measures than with nighttime symptoms; this could possibly be explained by the cognitive model of insomnia (Harvey, 2002), which postulates that repetitive thought processes (e.g. catastrophizing) have a direct influence on arousal and distress but only an indirect impact on sleep. Also, both ICS subscales were correlated with daytime impairment; the daytime catastrophizing subscale was somewhat more strongly associated with daytime impairment than the nighttime catastrophizing subscale. These latter findings were expected based on that the ICS was constructed to tap into nighttime and daytime catastrophizing.

This discussion must be considered in terms of several limitations. One initial limitation was the moderate response rate (47.2%), and the finding that respondents were likely to be older than non-respondents, which might have implications because age was significantly, negatively related with the two ICS subscales. However, only age and gender were considered in the attrition analysis, which might be a further limitation since other sociodemographic factors have been shown to be related to insomnia (Ohayon, 2002). Second, this study was cross-sectional, meaning that causal relationships cannot be deduced. Third, all measures were based on self-report which relies on the assumption that individuals can accurately report the processes assessed. This might have provoked a sleep category misclassification, with, for example, non-complaining poor sleepers having been categorised as belonging to the normal sleep group and vice-versa. However, we note that we used a sleep disorders screening instrument. Also, the research diagnostic criteria for insomnia define the condition as a subjective complaint. A replication with a sample of patients diagnosed on a semi-structured interview is nevertheless warranted. Although quantitative criteria are receiving research support as a diagnostic marker for insomnia, a limitation is nevertheless that our use of clinical cutoffs for initial, middle, and late insomnia (31 min or more; Lichstein et al., 2003) might have resulted in misclassification. It is also important to emphasise that self-reporting on nighttime symptoms, such as initial insomnia, is only reasonably correlated with more objective measures (Vallières & Morin, 2003), which hampers the conclusions from this study concerning associations between the ICS and nighttime symptoms. Another related imitation is that it is likely that response bias may have an impact on the findings concerning the ICS. More specifically, it is possible that participants with higher levels of insomnia catastrophizing respond differently to other questions as well (e.g. nighttime symptoms, arousal, and anxiety), relative to those lower in catastrophizing. The observed associations in this study may thus partly depend on how our participants responded to the ICS. Using objective measures (e.g. actigraphs for assessing sleep) in future studies would likely reduce the risk of response bias. Another related measurement issue in the current study concerns the HADS, which was used as an index for anxiety; based on documented structural, conceptual, and psychometric problems with the HADS (Coyne & van Sonderen, 2012), the correlations between the ICS and anxiety in the present study should be interpreted with caution.

Future research on the ICS is needed in a number of areas. The ICS subscales were not examined for test-retest reliability nor treatment sensitivity. Future research is also warranted on the inter-relationships between catastrophizing with worry and rumination within the context of insomnia examining whether these thought processes are distinct or overlapping constructs. Since a catastrophizing scale (CTIS) has already been validated, an investigation of the psychometric similarities and differences between the ICS and the CTIS is warranted. Finally, research is also needed on whether the ICS subscales predict insomnia symptomatology longitudinally or act as mediators in the management of insomnia, particularly so for cognitive behavioural therapy. In the area of chronic pain, it has been shown that catastrophizing is predictive of pain and disability within a treatment setting and mediates treatment outcomes (Wertli et al., 2014).

Taken as a whole, the preliminary indications are that the ICS is a psychometrically-sound scale for the assessment of insomnia-related catastrophizing. This might pave way for using the ICS as an assessment tool in clinical practice and research settings.

Acknowledgments

We would like to express our gratitude to the Swedish Council for Working Life and Social Research for financial support and to Maria Tillfors at Karlstad University and Steven Linton at Örebro University for comments on the ICS.

Disclosure statement

No potential conflict of interest was reported by the authors.

Additional information

Funding

This work was supported by the Swedish Council for Working Life and Social Research.