Exploring the content validity of the Chalder Fatigue Scale using cognitive interviewing in an ME/CFS population

ABSTRACT

Background

The Chalder Fatigue Scale, also known as the Chalder Fatigue Questionnaire (CFQ) is a Patient Reported Outcome Measure (PROM) comprising 11 items designed to measure physical and cognitive fatigue. It is widely used with people with myalgic encephalomyelitis/chronic fatigue syndrome (ME/CFS). There is no published evaluation of the content validity of the CFQ.

Objectives

To elicit information regarding the cognitive processes undertaken by people living with ME/CFS, when completing the CFQ to allow examination of the CFQ’s content validity.

Methods

A qualitative study utilising semi-structured cognitive interviewing. All data were collated according to the CFQ item but some general criticisms of the content validity of the CFQ were also identified.

Results

The CFQ currently consists of one item clearly related to physical symptoms (1.6), four items clearly related to cognitive function (1.8, 1.9, 1.10, 1.11) and one item relating to fatigue (1.5) which could be interpreted as cognitive and/or physical fatigue. The other five items have been identified by participants as lacking clarity (1.1, 1.7), relating to behaviour not symptoms (1.2, 1.4), or relating to sleepiness not fatigue (1.3).

Conclusion

Participants provided a wealth of insight into the challenges related to relevance, comprehensiveness, and comprehensibility of the CFQ, indicating that revision is required. This strengthens the case for participation of people with lived experience at all stages of PROM development. There is a need for an assessment tool/PROM for clinical and research use ME/CFS which has undergone content validation involving people living with ME/CFS.

KEYWORDS:

• Myalgic encephalomyelitis
• chronic fatigue syndrome
• content validity
• Chalder Fatigue
• PROM
• COSMIN

Background

Myalgic Encephalomyelitis/Chronic Fatigue Syndrome (ME/CFS) is a disabling condition with a wide range of possible symptoms [1–3] with an uncertain but frequently poor prognosis [4]. The paucity of high quality prognostic data is complicated by the different diagnostic criteria used and the lack of consensus about how to define recovery [5]. ME/CFS is characterised by fatigue in addition to unrefreshing sleep and/or sleep disturbance, cognitive difficulties, and post-exertional malaise [6]. It can affect children, young people, and adults [7] and is more common in females than males [8]. The prevalence of ME/CFS is between 0.2% and 0.4% in the UK [9] and estimates of the financial burden to the UK indicate a combined direct and indirect cost of £6.4 billion [10].

Despite a growing body of research into ME/CFS the aetiology and underpinning mechanisms are only partially understood [11]. Evaluating treatment in ME/CFS is challenging, in part because individuals are impacted in a range of ways [12]. With the current absence of biological markers, diagnosis and monitoring relies heavily on patient self-report. As fatigue is a core symptom, it is essential to have a reliable way of measuring fatigue severity.

The Chalder Fatigue Scale, also known as the Chalder Fatigue Questionnaire (CFQ) is a Patient Reported Outcome Measure (PROM) designed to measure physical and mental fatigue [13].

The CFQ consists of 11 items, phrased as questions, including lack of energy, sleepiness, tiredness, lack of strength and difficulty with concentration and memory. Responses use a Likert scale asking about the severity/intensity of symptoms compared to ‘usual’, defined as premorbid. Scores range from 0 to 3 on each item(0 = less than usual, 1 = no more than usual, 2 = more than usual, 3 = much more than usual) giving a highest score of 33. Alternately the scale can be used bi-modally with ‘less than usual’ and ‘no more than usual’ scoring 0 and ‘more than usual’ and ‘much more than usual’ scoring 1, giving a total score from 0 to 11 with 4 or more representing caseness [13].

The CFQ is widely used in the ME/CFS, other clinical and non-clinical populations and has undergone quantitative validity testing in multiple populations [14–18]. However, several problems with the CFQ have been identified. The bimodal scoring has been shown to have a low ceiling and discriminated poorly between people who rated themselves as moderately or severely affected [19]. The CFQ was not able to discriminate between a sample with CFS and a sample with primary depression [20], A comparison with the Fatigue Severity Scale (FSS) found that the FSS offered increased specificity and was a more accurate and comprehensive measure [21].

All PROMs are susceptible to multiple sources of bias such as the social desirability of responses, variations in introspective ability and differences in interpretation of questions or response options [22,23]. To develop reliable and valid measures, sources of bias must be identified and managed wherever possible. The ‘Consensus-based Standards for the Selection of Health Measurement Instruments’ (COSMIN) initiative aims to improve the selection of PROMs in research and clinical practice, by developing standards and criteria for evaluating their measurement properties [24]. The COSMIN approach to content validity identifies three aspects: relevance, comprehensiveness, and comprehensibility. There is no published evaluation of the content validity of the CFQ.

Cognitive interviewing is an established method of identifying the comprehensibility of PROMs [25,26] which explores the cognitive processes that responders undertake when answering a question. This allows identification of whether the responder has interpreted a question in the manner it was intended by the researcher [27]. Willis (1999) describes four processes when answering a PROM item:

• Comprehension of the item

• Retrieval from memory of relevant information

• Decision processes

• Response processes

Each process involves multiple cognitive steps: both conscious and unconscious. Willis (1999) describes two main techniques for cognitive interviewing:

1. ‘Think-aloud’ interviewing: the responder is instructed to ‘think aloud’ as they answer the PROM questions.

2. Verbal probing: after asking the PROM question and obtaining a response from the interviewee, the interviewer then asks for other specific information using ‘cognitive probes’.

Cognitive interviewing is useful in PROM development and when improving existing PROMs, by highlighting problems and guiding improvements over multiple rounds.

Study aim

The aim of this study was to use cognitive interviewing to elicit information regarding the cognitive processes undertaken by people living with ME/CFS, when completing the CFQ to allow examination of the CFQ’s content validity.

Ethics

Ethical approval was given on 11 January 2018, REC reference 17/NW/0726, North West – Liverpool Central Research Ethics Committee.

Method

A qualitative study utilising semi-structured cognitive interviewing. Assistant psychologists working in an adult ME/CFS service conducted face-to-face interviews in a clinic room. They gained signed consent and collected demographic data. The cognitive interview method was explained to the participant, and they were trained using a brief, standard ‘think-aloud’ cognitive interview training technique [25]. The CFQ was administered, and a mixture of scripted and spontaneous verbal probes were used concurrently to elicit more detail about their response processes. Scripted verbal probes included ‘Are these instructions clear?’ and ‘read each question aloud before going on to think aloud while answering the question’. Unscripted probes were used to ask questions about their comprehension/interpretation of the question (this may involve asking the patient to repeat the question in their own words), recall of information to answer the question, ease/difficulty of answering the question, and the certainty of their answer.

Data input and analysis

Interviews were recorded, transcribed verbatim and anonymised to maintain patient confidentiality. All data was collated according to the CFQ item. Thematic analysis was intended but responses were too heterogenous to report as overarching patterns, so the majority of responses relating to more individual experiences are presented separately in the findings. Some were grouped if there was overlap or duplication of responses such that duplicate quotes could be removed. All authors reviewed the responses and groupings and agreed that no further reduction was possible without losing key findings.

Study setting

This study was undertaken in a specialist outpatient service providing assessments and support for adults living with ME/CFS. Referrals were made to the service from Primary Care once a fatigue screen of blood tests was undertaken.

Sample and recruitment

Participants were recruited from the population of patients with a confirmed diagnosis of ME/CFS assessed within the service. Patients were required to meet both the Fukuda et al. [28] criteria and the NICE [29] criteria at assessment, so post-exertional malaise (PEM) was a necessary criterion for diagnosis. Participants were recruited from an introductory seminar about ME/CFS. They were provided with a copy of the patient information sheet and consented to be contacted to arrange the interview and inculsion/exclusion criteria were confirmed (Table 1).

Table 1. Study inclusion and exclusion criteria.

Inclusion criteria	Exclusion criteria
Participants were assessed by the Bristol ME/CFS Service with confirmation that their diagnosis of ME/CFS met the Fukuda [28] criteria and NICE [29] criteria. Minimum age of 18 years with no upper age limit. Able to attend an outpatient appointment.	Participants who are unable to speak the English language fluently were not able to take part in the study as misinterpretation of questions due to limited language capabilities was a potential confounding variable. Individuals with any cognitive impairment other than that resulting from ME/CFS were excluded from the study.

Results

Sample

Thirteen participants took part (see Table 2) before the COVID-19 pandemic stopped delivery of the face-to-face seminars where participants were recruited: sufficient data had been collected to constitute a round of cognitive interviewing [26].

Table 2. Participant demographics.

Characteristic	Number of participants (n)	%
Gender
Female	11	84.62
Male	2	15.38
Age bracket
18–30	1	7.69
31–40	4	30.77
41–50	6	46.15
51–60	1	7.69
61–70	1	7.69
Ethnicity
White	13	100
Duration of illness (months)
6–11	3	23.08
12–23	4	30.77
24–35	2	15.38
36–120	2	15.38
121–240	2	15.38

The CFQ begins with brief instructions (Figure 1).

Figure 1. The instructions presented at the start of the CFQ.

The instructions presented at the start of the CFQ.

Participants identified a range of problems related to the instruction ‘compare yourself to how you felt when you were last well’. Many participants had been unwell for a long time and had difficulty remembering life before:

It is probably challenging to remember when you were last well when you’ve had the illness for 20 years. (Participant 9)

I’ve got this halo figure of how I was before in my head. (Participant 13)

One doubted their ability to make the comparison:

So you’ve got to forget about when you were tired and base this question on when you were feeling well, that would throw me. (Participant 8)

These problems led some participants to find ways of making comparisons which were not what the CFQ intended, such as comparing to ‘a better time’ during their life with M.E.:

What happens if you can’t remember the last time you felt well? If you are not sure what is well and what is normal and what is M.E. Immediately you are like … what am I comparing myself to? So, I would go when is a better time that I feel and compare myself to that rather than actually feeling well per se. (Participant 12)

The emotional cost of recalling how a participant felt when they were last well was also reported:

I’m not sure people want to think back to when they were last well as that can make you feel quite sad, about the life that you had and the life that you have now. It is just a bit rubbish to think this is what I was doing, and this is what I am doing now. (Participant 4)

Question 1. Do you have problems with tiredness?

Several interpretations of ‘tiredness’ were identified. Uncertainty about the intended meaning affected some participants’ confidence in responding:

What do you mean by tiredness? Is it mental tiredness or physical tiredness or do you mean fatigue? Are you lumping it as the same thing?

Researcher prompt: Is this an easy or difficult one to answer?

Participant: I’d say it is quite difficult in that it is quite vague, and I would be a bit worried that I am not answering it in the way you mean it to be. (Participant 3)

One separated tiredness and sleepiness from fatigue:

Yeah much more than usual. Although I can separate tiredness and fatigue. I still get more tired than previously I would. I need more sleep. (Participant 13)

Some reflected on experiences of unrefreshing sleep and a complex relationship between ‘tiredness’, an increased time in bed, and increased daytime sleep:

Just exhaustion, just constantly waking up tired. (Participant 2)

I don’t want to get out of bed in the morning, but I do, and I need to go to bed a lot earlier than I did. (Participant 1)

If you go on average during the day, I usually have to have a nap, so I would say more than usual. (Participant 5)

I do go to bed early because I’m tired, but I am aware that it is a mental fatigue rather than just bedtime. (Participant 6)

Despite the heterogenous interpretations, all selected a single response, and some reported it to be easy to complete.

Question 2. Do you need to rest more?

Participants noted that there are different types of rest, and that rest for a person with ME/CFS might be different from rest for a person without fatigue. Rest could mean stopping in the middle of an activity, changing position (e.g. sitting down or lying down), doing nothing, or even sleeping:

I need to rest doing simple things, like walking for a minute or two. If it is longer than a 10-minute walk I would stop a couple of times. (Participant 2)

The answer is pretty easy, I do. I guess people will have a different interpretation of what rest means, and whether that is simply sitting down or whether I sort of count that as sleeping …  … it could potentially cover everything from sitting down to just stopping what I am doing. (Participant 9)

For me, resting is lying in bed doing nothing. Which is how I’ve spent several days in the last month. Although rest is probably perceived as reading a book or watching TV. (Participant 13)

Participants explained that it can take time to learn to rest, and that there is a difference between planned rest as a coping strategy compared to enforced rest (‘crashing’) due to PEM:

If someone has problems with tiredness and they have been referred here I am guessing they need to rest more. Whether they do rest more day to day and crash is another matter. Anyone with this condition if they are going to treat it sensible needs to rest more than how they used to. (Participant 4)

I find that I need to put my feet up in the afternoon and I find that If I don’t do that, I feel really tired. (Participant 1)

Although participants explored different meanings of rest and acknowledged that the measure is not sensitive to the different types of rest, all participants endorsed one of the responses.

Question 3. Do you feel sleepy or drowsy?

Some described their experience of fatigue being distinct from sleepiness or drowsiness, which meant that they had difficulty interpreting the item and reaching a decision about how to respond:

I don’t know, that’s a silly question I think because I always feel tired. I wouldn’t say I feel drowsy, I could always sleep. More than what or less than what? I would say no more than usual because I wouldn’t know what to answer that honestly. (Participant 8)

For one, the dominance of fatigue over sleepiness in their lived experience influenced their interpretation of this question. Nevertheless, they endorsed ‘much more than usual’ because it seemed the only way to capture the severity of their problems:

Well, that doesn’t really get to the bottom of the fatigue that you feel … sleepy and drowsy is a thing normal people have, it’s not a fatigue thing. Yes, I feel sleepy and drowsy, but I also feel so much more than that. So yes, I feel sleepy and drowsy but that’s not getting to my real problem.

Researcher probe: What would your answer be for that one?

I feel permanently tired, I’ve got brain fog and I’d so much like to have a sleep but that’s not my day, that’s not how I have to live my days. I don’t live my days thinking oh if I had a nap I’d feel great now. I feel crap all the time. So do you feel sleepy or drowsy … yes much more than usual. I’ve felt like this for a very long time … I almost have to pick much more than usual because that’s the only one that gets close. (Participant 13)

One suggested that it could be asking multiple different elements: the feeling of the need to sleep, feeling unrefreshed from sleep, tiredness, the chances they could sleep, and drowsiness.

Do I feel like I need to sleep, or do I feel like I haven’t fully woken up? So, to me, do I feel tired? Yes! do I feel like I can sleep? No! Do I feel drowsy in that I don’t feel fully awake and feel unrefreshed? Yeah much more than usual! (Participant 2)

One participant reported having recently learned to distinguish the physical experience of sleepiness (typically associated with heavy eyelids, a nodding head and yawning) from physical fatigue (a whole-body experience) and that this distinction would now influence their response.

I’d say more than usual, it’s not really a sleep-drowsy thing, I think I have more like an all over body fatigue, so I guess yeah … it probably depends on how people interpret it I suppose, because you could kinda see that as sleepy, it’s just learning the difference between sleepy and fatigue has been a fun thing to learn. (Participant 7)

Others did experience sleepiness associated with their ME/CFS, recognising that this was a key part of their experience of the condition:

Absolutely that was one of the hallmark things that happened after I had ME so I would put much more than usual. (Participant 9)

One participant identified sleepiness at bedtime, which led them to endorse this item but the context of the sleepiness didn’t seem pathological:

I often do, when I go to bed because I read and I keep dropping the book. (Participant 1)

Temporal variation in sleepiness was raised as being relevant to decision-making about responding:

I’m thinking is that all the time? Yeah, I would interpret that as all the time and yes, it’s quite clear and I would have answered more than usual. (Participant 11)

I feel sleepy at different times. I think that would be the thing to say. So more than usual probably not much more over the last month. (Participant 12)

These quotes also raise a question about whether the response ‘more than’ means more frequent, more severe, or both.

One participant noted concerns about the consequences of completing the item. They considered whether a high score would cause concerns relating to driving safety.

There is a part of me thinking … are they trying to take my licence away? (Participant 4)

They also considered if a lower overall CFQ score would prohibit later provision of care:

I would think that people doing these would be slightly more worried about not getting in (to the Service) if they don’t put more than usual for all of them. They are quite desperate for help and to find a way through this condition. I did answer mine but did think … is this a way of filtering and getting rid of people so the money doesn’t have to be spent on them? (Participant 4)

Question 4. Do you have problems starting things?

Some were uncertain whether this item was asking about motivation to start activities or having the energy to do them:

Maybe it means, do you put things off? Something about motivation? It’s not clear whether it means from a motivational perspective or physical energy perspective? (Participant 3)

For another, the energy and motivational components appeared intertwined:

Starting things, as in stuff to do and I think I don’t have the energy to do it, so I won’t get it finished. I know I do have problems with that because I’m not gonna have the energy for it. (Participant 6)

This question prompted one to think about their difficulty finishing tasks:

Just asking if things are a bit harder to get going, I’d say finishing things is more of a problem. Starting things too … some people can start lots of things and then feel too exhausted to finish it off. (Participant 4)

For some, difficulty starting things was related to cognitive activity before initiating a task.:

If I am thinking about projects, things that I thought I really need to do. Then yes, I spend a lot of time thinking about things before I start it. So, I guess yeah more than usual. (Participant 2)

Whereas now that seems a million miles away, to start the engine in my head to get things done. (Participant 13)

The lack of clarity about which kind of activities were being considered also made it difficult for participants to answer, as easier tasks could be started and harder tasks were not started at all:

I wouldn’t be completely certain; I wouldn’t be quite sure what kind of things do they mean like thing little things or bigger things? (Participant 2)

Does it mean big things or little things? Some things I can just do and other things I haven’t done at all. (Participant 12)

Increased fatigue following task completion was identified as causing apprehension which affected task initiation:

I would interpret that as I have to do some chores, I have got to go out and do something, I’ve got to do some work. the actual act of starting it, but I’d put it off for quite a while, more than I used to because I know it’s gonna make me tired, so yeah, I would definitely do that one. (Participant 7)

One identified the focus of the question as procrastination, but they saw this as a personal trait independent of their M.E.:

I’m guessing it’s talking about procrastinating, and I am relatively well known for that but then I probably was before I was ill so I’m not sure if it’s a significant marker for M.E. (Participant 9)

There was also evidence of a relationship between the variable and cyclical nature of the condition over time, and the impact on being able to start activities:

So do I have problems starting things … well that depends on …  … what part of the condition I am in as to whether that’s true. If the question is do you have problem starting things when it’s bad, then yes much more than usual is the answer. But if I’m having a good day or a good few weeks then no, I don’t have any problems starting things. (Participant 13)

Question 5. Do you lack energy?

This is the first item that provided little difficulty for most. Examples were offered about the functional impact of this lack of energy which illustrated their experiences:

It is a struggle to physically get moving. So, I can go to work and sometimes I am almost in tears. So yeah, I lack energy. (Participant 5)

Yes I do lack energy as I’m not able to do everything I used to do before. (Participant 1)

I think that’s a very clear one to interpret and I absolutely do struggle with energy levels much more than I did before I was ill. (Participant 9)

One noted that they were able to identify a lack of energy due to their health which was separate from that which might be due to other life circumstances. When asked to paraphrase the question, they responded:

Have you got enough energy that you would expect to have considering your circumstances? Because I would say I do take into account when someone asks me that, I do have two young children and that in itself is quite tiring. I would expect that I am more tired than some people. This is more of a holistic question, if you take those things out of there do you think you have less energy than you think you should have? (Participant 12)

One identified a dual meaning for the term ‘energy’: firstly, the physical and cognitive capacity to be active, and secondly the motivation or ‘will power’ to be active:

When I’m feeling bad, every day is a succession of mental hurdles that I push myself over to do the things that I know need to get done. So yes, I lack energy, but I don’t lack will power. (Participant 13)

One questioned the role of this item in a questionnaire administered to patients attending a fatigue clinic:

It is that question again, you wouldn’t have got to this place without lacking energy. But maybe that needs to ask to make sure the service can survive. But it just feels a bit daft from a filler inner point of view to ask that question when you would have been seeing your doctor for a while. (Participant 4)

Question 6. Do you have less strength in your muscles?

There was general agreement that this item was relevant, but there was complexity. Two participants offered specific examples of muscle weakness and its’ impact on function:

Do I struggle with my muscles, having less energy in them? To me this makes me think of the fact that I struggle to hold my arm up above shoulder level. My neck 80% of the time I need to support my neck with my hand. I struggle holding my head up without it aching. I feel like to me the idea of having less strength in your muscles is like feeling like a baby. You are weak and can’t do the things you’re trying to do. (Participant 2)

That’s a really marked one; certainly in my limbs, it is noticeable that I’m not able to carry as much as I use to or that I’m not able to walk as much as I use to, so much more. (Participant 9)

One took a different position, acknowledging the item was easily understand but that muscle weakness might not always be perceptible to an individual:

More than a day ago I would have answered that I didn’t have less strength than usual in my muscles. I happened to be part of a study yesterday that tested my muscle strength, it showed that my muscle strength is low, in a way that I hadn’t perceived myself.

(Participant 3)

The impact of increased rest and reduced activity on muscle strength was identified, with one participant identifying visible muscle atrophy over a 2-year period. Another made sense of this secondary deconditioning:

If you are lying down a lot more or sitting down a lot more, you’re moving a lot less. You’re gonna have less strength in your muscles, it’s a bit of a given. (Participant 4)

A more complex experience of muscle strength was also described, with one finding it difficult to tease out the effect of deconditioning from the ME/CFS. The conclusion was that both were relevant, and that the item could be answered by taking both into account:

Yes, but I don’t know if that’s from the CFS or the inactivity. I think that could be taken two ways really. Like actual physical strength from the fact you haven’t been doing … energy has definitely decreased. But also, the energy that you feel in your muscles feels a lot less because of the fatigue. I think that could be taken one of two ways but yes to both of them: I suppose I wouldn’t have trouble answering it. (Participant 7)

This uncertainty around the underpinning mechanisms of the problem was taken a step further by highlighting a problem with the ‘engine’ which drives the muscles:

I don’t think my problem is because of my muscles … it’s the engine not the muscles. I don’t see it as a muscle thing in my legs or whatever I don’t think that’s necessarily … I can’t do stuff not because I don’t have the capacity in my legs or arms it’s because I haven’t got the drive to do it. (Participant 13)

Question 7. Do you feel weak?

Some participants identified ambiguity around the dual meaning of ‘weakness’, stating that it could relate to either physical or mental weakness, and found it difficult to clarify which aspect was being asked about:

I suppose that could be taken two ways. I took it as physically, but I guess some people could take it as mentally. I think I carried on from the muscles … do you feel weak? as a continuation from that but yeah, I suppose that could be taken in several ways. (Participant 7)

It was also identified as meaning the same as the previous item which asked about loss of muscle strength:

Yeah again, to me, do I feel weak, do I have less strength that’s the same question that’s how I perceive that question. (Participant 8)

In contrast, one considered the possibility that the item could refer to stamina, i.e. being able to keep walking:

I’d say that is a little bit open, because you’ve got in terms of the muscles or I’d see it as how far I can walk without having to sit down, I would interpret it as the second one …  (Participant 3)

The lived experience of ‘weakness’ was expanded on by two participants, indicating that the term has more complexity, and could be explored further:

It is that feeling of slight disconnect from when you are feeling strong and pick something up … when you are feeling quite weak … it’s like am I going to drop it. (Participant 12)

Absolutely, I would count a lot of other things in that as well to do with shaking and wobbling. I’m interpreting weak to cover quite a few factors in there, everything from opening jars to feeling strange when I stand up or walk. So, I would also count in that dizziness and potentially other things that’s not covered by feeling weak but I’m not sure. (Participant 9)

One participant considered the post-exertional nature of symptoms was not captured:

I don’t think I’d struggle to lift that any less today than when I would when I was well. My strength is my strength … the difference is within seconds I would have walked away in the old days and now I’d suspect I would feel it pretty quickly. (Participant 13)

Question 8. Do you have difficulty concentrating?

Some found it difficult to answer without an example of what they might concentrate on:

It is very hard to see what kind of thing you would be concentrating on, it is good when they give you examples. Concentrating on reading a book or watching TV or solving a problem. Passive or active concentration. (Participant 2)

Some generated an example of something they might concentrate on:

Yes, I know I do, because I’ve found trying to fill in forms and that takes me a lot longer than it would normally. (Participant 1)

I can’t read … I can’t sit and watch TV at home, I tend to just listen to audio books. Yeah, I’ve got no capacity to really get into stuff. When people are talking, I find it difficult. That is probably the thing that stops me going to work most. (Participant 12)

Some could answer straightforwardly, for example this interpretation allowed an easy answer:

That one is quite straight forward. Are you finding it hard to keep your mind on something or not. (Participant 12)

However, there was complexity identified, partly because one respondent identified difficulty concentrating but was not sure of the extent to which it was due to ME/CFS:

I do have difficulty concentrating and I’m never quite sure how much of that is due to ME. I expect this illness exacerbates. (Participant 9)

The same participant described complexity in responding due to being out of practice regarding concentration:

If you are used to working and used to being a certain way and you are taken out of that environment, and you don’t have any of that structure in place … my brain is out of practice of concentrating at the moment. (Participant 9)

One indicated a problem relating to the participants instruction to consider their problems in ‘the last month’ compared with a time they last felt well. They stated that the condition was cyclical for them and that they experienced relapses following physical exertion (PEM) which would significantly worsen their ability to concentrate. They were concerned that the questionnaire did not capture this variability in experience:

But to answer the question do I have difficulty concentrating … yes much more than usual when I’m having a bad period. Again, this isn’t reflecting the cyclical nature of what I’ve experienced which is over months at a time if I’ve caused myself a problem. For me it’s me causing myself a problem with physical exertion and putting me into a crisis or a flare up. That can last months … so it sort of needs to be phrased in terms of when you’re having a good time versus when you’re having a bad time not just when you last felt well. (Participant 13)

Question 9. Do you make slips of the tongue when speaking?

Some found this to be straightforward to answer, using personal experiences to illustrate this:

Yes, I do, because my husband keeps telling me I am, and I do find that I am. (Participant 1)

Yes, that is getting increasingly worse. I work in a call centre and my brain can say something completely different to what I want to say. (Participant 5)

Some also considered word-finding difficulties when responding and differentiated these from slips of the tongue:

It is not so much slips of the tongue, I don’t say the wrong word, I sometimes just can’t think of the right word to say. I think I get that more than I do slips of the tongue. (Participant 7)

Others were considering additional cognitive difficulties when responding to this item and they felt that the wording of the item trivialised cognitive difficulties:

Yes, but it’s more to do with my brain not operating the way that it should do. Lots of things don’t work in my brain the way that they should do. There’s an implication in there that a slip is a minor thing but it’s more of an unable to operate at the level I used to. I can’t think the way I used to. So yes, slips of the tongue trivialises something which is much deeper, the cognitive problems which you get. Maybe that’s supposed to be an indicator for it, and this is a litmus test for saying how is your brain fog. But it seems to trivialise the problem. (Participant 13)

Finally, one who had pre-existing problems with dyslexia and dyspraxia found it to be difficult to distinguish the effect of these conditions from ME/CFS:

Probably so no more than usual, as I have always done that. I have dyslexia and dyspraxia. Maybe it is more than usual. Because I found my dyspraxia symptoms are much worse when my chronic fatigue symptoms are worse. (Participant 2)

Question 10. Do you have problems thinking clearly?

Some participants noted the similarity between this item and item 8 relating to concentration:

There’s nothing wrong with it but I guess concentrating and thinking clearly are not too apart. (Participant 4)

I guess that’s kind of the same thing as concentrating, I would view that as the same thing as concentrating. That’s the only time when I need to think clearly is when I’m concentrating on something otherwise it’s fine to be fuzzy up there.

One recognised a difference between thinking clearly and concentrating, and that they were considering this when responding:

This one is different than the concentrating one. My first thought is that … oh what’s the difference between that and that, I would have thought about what they are measuring and read that as concentrating is staying on task and being able to follow what you are doing. Thinking clearly is more finding it hard to think about the process that needs to happen for something. (Participant 12)

Others raised the issue of ‘brain fog’ and the extent to which this experience overlapped with or was separate from problems with thinking and concentration. Some were satisfied that they were similar:

I’d say much more than usual. I feel a bit foggy. (Participant 2)

Things like brain fog, that cloud that comes down and you can’t think through it, you lose words. (Participant 12)

The variability of the extent of the problem from day to day was noted, for example if stress levels were higher:

Um, it depends on my stress levels, if I have a lot going on I can go round in circles, so I would put more than usual but it does depend on the day. (Participant 5)

Very much gets worse if I’m stressed, it is noticeable since I’ve got ill. (Participant 9)

Question 11. How is your memory?

All identified this as a question that was relevant to their personal experience of ME/CFS. When considering if their memory was affected, one participant stated:

 … for an ME sufferer it will be. It is almost like asking if you have ME or not. (Participant 12)

However, participants differed in their experiences regarding which aspects of their memory were affected (e.g. short-term versus long-term memory) and they described variability in the types of information they tended to forget:

That’s clear, you have your short-term memory and your long-term memory. I’d say that my long-term memory hasn’t been affected as far as I can tell, my short term has so I would answer on the basis of that, more than usual. (Participant 3)

 … is terrible and it covers everything, everything from things that happen quite a long time ago to short term memory I really struggle recalling numbers and facts: not so much personal facts, more on the work side. (Participant 9)

It is worse than usual, I forget people’s names, forget what I’m doing, I can forget where I put something. (Participant 5)

Um … for what? My memory seems to be affected in different ways and some people seem to think about that in short term memory loss but it’s not really that either. I get blocks on certain things, but other things come to my mind very easily. Stupid things that I never thought i would ever remember in my current state seem to come to me without problem. So memory has lots of facets to it and the question trivialises it. You could go into more detail about the different types of memory: ‘do you remember things from today?’, ‘do you remember things from yesterday?’, ‘do you remember things from when you were a child?’. I’d put worse than usual because there are some facets of my memory that are okay. (Participant 13)

Discussion

The cognitive interviewing approach identified several challenges to the content validity of the CFQ. The discussion will explore the general issues first then turn to those relating to specific items.

One challenge relates to the initial instruction: ‘If you have been feeling tired for a long while, compare yourself to how you felt when you were last well’. Most participants had longstanding ME/CFS so were being asked to recall how they felt many years ago: they doubted their ability to do this accurately. To situate this issue in a clinical context, all patients presenting to specialist clinics for ME/CFS within the NHS in England will have experienced symptoms for a minimum of 3 months to have been considered for referral in accordance with the NICE (2021) Guideline, with the median duration reported as 3 years [30].

Another challenge is that this instruction seems to offer a choice: that only those who had longstanding symptoms should compare themselves to when they were last well. In contrast, the questionnaire is set up to capture any fatigue problem lasting more than a month, so a less confusing instruction could remove the phrase ‘If you have been feeling tired for a long while’.

Participants highlighted that questions can be difficult to answer because of the variable nature of ME/CFS, particularly in relation to PEM. Over the course of the ‘last month’ some had experienced a range of symptom severities and considered that it is important to represent this variable experience. As it stands, the CFQ does not allow participants to represent their variable experience over the past month, and the impact of PEM. In addition, the questionnaire does not capture information about the cyclical nature of the condition over a longer period. An additional consideration is that if a respondent experiences some flare-up and some relatively better days in the last month, accurately retrieving information from memory and deciding upon a response option to represent an average of this, would clearly be difficult. A shorter time frame such as the past week may simplify these issues.

The response options also raised questions for some participants who indicated that endorsing the response items ‘more than usual’ or ‘much more than usual’ might indicate an increase in severity, or frequency, or both. Clarity regarding the response options would allow for more consistent interpretation and subsequent responses. There was also an indication that some participants coped with uncertainty about the meaning of some items by considering how bad their ME/CFS had been and opting for the response which best matched their overall experience.

Some expressed uncertainty about the purpose of the questionnaire, particularly in the context of an assessment within a specialist service where some of the symptoms would be expected. Concern was expressed about whether a certain score would be required to access the service, as a way of rationing resources, or might impact on service funding. Clarity about the purpose of the CFQ (or any assessment tool) would help to address this, with reassurance to the participants that their responses will be kept confidential within the service, will not affect the funding of the ME/CFS service, and will not affect the level of therapeutic input that they receive.

The discussion will now focus on each item. Question1 asks ‘Do you have problems with tiredness?’: this is one of several items that participants interpreted in multiple ways. Participants’ lived experience of ME/CFS gave them a more nuanced understanding of different types of fatigue which the word ‘tiredness’ did not represent. Participants identified subgroups including unrefreshing sleep, a need for increased sleep, exhaustion, mental fatigue, physical fatigue, and post-exertional malaise and their interpretation affected their response. More consistent interpretation of the question could be achieved if these heterogenous experiences were captured in separate items.

Item 2 ‘do you need to rest more’ asks about a behaviour in response to symptoms, not a symptom. Participants offered a nuanced understanding of the role of rest. A clear separation was made between ‘needing to rest’ because it was enforced by increased symptom severity (colloquially known as ‘crashing’) and a decision to use planned rest as a self-management strategy. The aim of strategic, planned rest was to reduce the incidence and severity of PEM which otherwise can lead to enforced rest. The wording of the questionnaire item allowed for both types of rest to be considered in responding to this item. However, not all respondents would consider both types of rest, indicating a challenge with consistent comprehension between respondents of what is being asked. A questionnaire redesigned to assess coping strategies could include examples of these different types of rest, to reduce heterogeneity in the responses but a redesigned questionnaire focused just on symptoms should remove this item.

The responses elicited by item 3 relating to sleepiness and drowsiness, indicate that this item does not represent the experiences of some people living with ME/CFS and is open to misunderstanding. Further description about what could be considered as sleepiness or drowsiness could aid respondents interpreting the question more consistently. The importance of the timing of sleepiness could also be considered if the item were to be redesigned (e.g. do you feel drowsy or sleepy in the daytime). Given the extent of the variability in these experiences, evidenced in other research [31], it may be more appropriate to omit this item and use a questionnaire designed to enquire about excessive sleepiness (somnificence) such as the Epworth Sleepiness Scale [32].

Some participants identified problems ‘starting things’ (item 4) while others reported greater problems completing things. This suggests that the addition of a separate item asking about difficulty finishing activities could ensure that all aspects of the condition are being measured (increasing content validity). Participant uncertainty about what ‘things’ should be considered when responding to this item, indicates that participants are likely interpreting the question differently from one another, posing a challenge to the validity of the CFQ. However, the vagueness of the question does allow the participants to consider the individuality of their experience and consider the ‘things’ that are most salient to them. It may also mean that individual participants may interpret this item differently from one occasion to the next, affecting test–retest reliability. Offering some further information about the types of activities to be considered may help increase consistency of responding. Multiple items assessing difficulty starting things due to different factors could be one approach to improving the measure, with a cost of increased participant burden. As it is not a symptom, it may be simpler to remove this item.

Item 5 asked ‘do you lack energy?’ and was the first item which was not problematic. However, responses indicated that this item could have been more nuanced if it enquired separately about physical energy and mental/cognitive energy.

Item 6 ‘do you have less strength in your muscles?’ generated a range of responses. Some were able to endorse this item with little difficulty. Some participants questioned the cause of the weakness when responding: Some could identify secondary deconditioning in addition to the impact of ME/CFS on strength, another experienced the loss of strength not as muscle weakness but as a failure of the ‘engine’ which drives the muscles. The role of both peripheral and central fatigue has been identified in research [33,34] which can lead to a loss of stamina, as well as strength. Further investigation of the experience of muscle weakness, in collaboration with people living with ME/CFS, would likely provide better understanding that could allow the question to be revised to allow for consistent interpretation of what is being asked.

Item 7 asked ‘do you feel weak?’ and participants’ responses included uncertainty whether the item referred to mental or physical weakness. The responses which considered physical weakness raised different interpretations such as loss of stamina, dizziness, impaired control of movement, and PEM. The heterogeneity of interpretations indicates that instruction regarding what types of weakness should be considered may be useful to provide clarity to respondents and prompt them to consider the multi-dimensionality of weakness to improve consistency of interpretation between respondents.

In response to item 8 regarding ‘difficulty concentrating’, it was recognised that difficulty concentrating was contextual, and that examples would have helped, for example, concentrating when with people, when communicating, writing, or reading. Providing contextual information could prompt respondents to consider their specific experience of concentration difficulties to aid more consistent interpretation. However, it is possible that the vague nature of the question allows the potential for a broad range of respondent experience to be captured representing individual differences.

Item 9 asked about ‘slips of the tongue’ and some respondents could identify this problem. Others suggested that word-finding was a more significant problem than making slips of the tongue, and that ‘slips of the tongue’ only present a minor aspect of a more extensive cognitive dysfunction. One participant said that they felt asking about ‘slips of the tongue’ trivialised the cognitive difficulties associated with ME/CFS, although there are other items which examine other cognitive aspects of their experience. These reports indicate that revision is required.

Item 10 asked ‘do you have problems thinking clearly?’ and participants were divided as to the extent to which this overlapped with question 8 which asked about difficulty concentrating. There was a suggestion that concentration (8) might refer to focussing on external inputs whereas ‘thinking clearly’ (10) might relate to one’s own thoughts. Some mentioned the colloquial term ‘brain fog’ which could be a cover-all phrase for cognitive difficulties. The variability of the problem over time and the influence of contextual factors such as stress made it difficult for some to respond. This is clearly an important issue for participants and more exploration of the lived experience of these complex issues may be required to inform an improved version of this item.

Item 11 asked ‘how is your memory’ and all participants identified this as a question that was relevant to their personal experience of ME/CFS. However, participants differed in their experiences of the components of memory which were affected. Having multiple questions asking separately about different types of memory difficulties (e.g. long-term and short-term memory difficulties) may make it easier for participants to answer.

Strengths and limitations

One limitation of this study is the small sample size. However, the number of participants is within recommendations for one round of cognitive interviewing [26] and was sufficient to identify many areas of difficulty when completing the CFQ which indicate that revision could improve content validity. Another limitation is the limited ethnic background of the sample. There was no specific data generation regarding content which might be missing from the CFQ, such as ME/CFS symptoms missing from the CFQ so the study therefore gleaned limited information about the comprehensiveness of the CFQ: an inductive qualitative study design would be required to optimise data collection about this aspect. There was no involvement of patients living with ME/CFS in the data analysis, however two authors have lived experience of fatigue: one ME/CFS and one cancer-related fatigue with PEM, who can be considered as insider researchers, offering additional sensitivity to issues raised by participants.

This was an unfunded study which highlights problems with the CFQ, but the research team have no further resources to undertake the significant further work indicated relating to CFQ revision. The researchers would welcome others to undertake this work, and to incorporate the findings into future measure development.

A strength of this study is that the participants were patients in an NHS service recruited following confirmation of the diagnosis. Therefore, the study has high ecological validity in that it explored experiences of a PROM in a clinical setting it has been used widely in.

Future research

If a revision of the CFQ is undertaken, then the instructions relating to the timeframe should be improved and the individual items should be revised in line with the suggestions in this paper and the COSMIN guidelines should be followed (24). If an improved assessment tool for ME/CFS is to be developed, then the purpose needs to be clearer. If the aim is to capture the consequences of ME/CFS or coping strategies used, then this would require a larger project which would require significant groundwork in qualitative research.

Conclusion

The CFQ measures some aspects of ME/CFS and provides limited information about experiences of ME/CFS. This study has highlighted multiple areas of difficulty that respondents encounter when responding to items which affect the validity of the measure.

The findings indicate that the CFQ consists of one item clearly related to physical symptoms (6), four items clearly related to cognitive function (8, 9, 10,11) and one item relating to fatigue (5) which could be interpreted as cognitive and/or physical fatigue. The other five items have been identified by participants as lacking clarity (1, 7), relating to behaviour not symptoms (1, 4), or relating to sleepiness not fatigue (3).

The participants provided a wealth of insight into the relevance, comprehensibility and to a lesser extent comprehensiveness, of the CFQ items. This strengthens the case for participation of people with lived experience at all stages of PROM development and supports the value of retrospective evaluations of construct validity of existing PROMs.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Notes on contributors

Peter Gladwell

Pete Gladwell is a Visiting Research Fellow at the Centre for Health and Clinical Research (CHCR) at the University of the West of England, a Clinical Specialist Physiotherapist at North Bristol NHS Trust, and Service Lead of the Bristol M.E. Service. He has collaborated extensively with the charity Action for ME, and has co-authored a peer-reviewed paper on the Action for ME 2010 rehabilitation survey. He is involved in the delivery of the Masters level fatigue course at the University of the West of England. His PhD investigated the use of TENS devices by people with chronic musculoskeletal pain. He has a particular interest in Patient Reported Outcome Measures, and the extent to which they match the lived experience of people with persistent health problems, especially as this relates to their experiences of specific health interventions which might have different outcomes from the focus of the PROM. He was involved in all stages of this paper except for data collection.

Matthew Harland

Matthew Harland was an Assistant Psychologist within the North Bristol NHS Trust ME/CFS service at the start of this research project, he wrote the research protocol and submitted the NHS ethics application. He has since completed Doctoral training as a Clinical Psychologist and he is currently specialising in Clinical Neuropsychology. Matthew has continued to work with the team to revise and edit the manuscript and in particular he revised the introduction and methods sections of the paper.

Aysha Adrissi

Aysha Adrissi was an Assistant Psychologist within North Bristol NHS Trust at the start of data collection, conducted some of the recruitment and interviews, and had a large role in data analysis. Aysha is currently engaged in Doctoral training as a Clinical Psychologist, and has continued to work with the team to revise the manuscript, in particular the findings and discussion sections of the paper.

Saskia Kershaw

Saskia Kershaw is an Assistant Psychologist within North Bristol NHS Trust and is also currently engaged in Doctoral training as a Health Psychologist at the University of the West of England. Saskia worked in the Bristol M.E. Service at the start of data collection, conducted some of the recruitment and interviews, and had a significant role in data analysis. She has continued to work with the team to revise the manuscript, in particular the findings and discussion sections of the paper.

Emma Dures

Emma Dures is a chartered psychologist based in Academic Rheumatology at the Bristol Royal Infirmary. She is interested in the interaction of psychological, emotional, and social factors with the physical health and well-being of people with inflammatory rheumatic conditions. Emma specialises in qualitative methods, often as part of mixed methods study designs. She works with national and international collaborators from nursing, medicine, patient involvement, psychology, and allied health professionals. Emma leads the Rheumatology Team at the University of the West of England (UWE, Bristol), conducting applied research to address questions from clinical practice. Their focus includes outcomes important to patients, the psychological impact of inflammatory rheumatic conditions, self-management support provision, fatigue, patient activation and nurse-led care. Collaboration with Patient Research Partners is integral to their research. Emma has offered academic support for the work since its inception and has offered constructive feedback and guidance during the drafting stage of the paper, including methodological advice and detailed editing.