Evaluating the usability of two salutogenic instruments on health and work experience, using cognitive interviewing

Figures & data

Table 1. Description properties and Cronbach’s alpha (CA) for the Salutogenic Health Indicator Scale (SHIS) and the Work Experience Measurement Scale (WEMS).

Questionnaire	SHIS	WEMS
Content with Cronbach’s Alpha (CA)	Twelve indicators of the individual’s health from physical, psychological, and social health dimensions:	Six areas (32 positive statements) related to experiences of work situation:
	1. Alertness	1. Internal work experience (0.82)
	2. Happiness	2. Supportive working conditions (0.85)
	3. Calm/relaxed	3. Autonomy (0.80)
	4. Sleep	4. Time experience (0.71)
	5. Concentration	5. Leadership (0.87)
	6. Ideas/creative	6. Process of change (0.89)
	7. Decision making
	8. Emotional balance
	9. Felt well
	10. Energy
	11. Function with other people
	12. Body function
	CA for the whole index was 0.92
Overall question	How have you felt in the last four weeks with regard to the following? In the last four weeks, I have…	Please enter your opinion on the following statements about your work situation.
		The further to the left you put your response tick, the more you agree with the statement.
Semantic differential	From 6 to 1 with a positive and a negative endpoint: e.g. felt happy, optimistic (6) to felt depressed, sad (1) and had lots of energy (6) to had very little energy (1)
Six-point Likert-type scale		From 6 to 1: Totally agree to Totally disagree.

Table 2. Overview of categories, subcategories and quotes by number, illuminating each subcategory in the result.

Category	Sub-category	(Nr)	Quotes
Interpreting	Difficulty	(1)	[…] when different work tasks shall be done, that I cannot decide as it is dependent on the patient; like I cannot decide when he/she should go to the toilet or is going to surgery; but of course I can decide on some tasks[…]but there are also laws and routines[…] (R4)
		(2)	I think my employer is investing in my health; that I do think as we get a lot of money for keep-fit activities, both for activities and massage […] (R3)
		(3)	The process of change is very unclear to me; I think both of reorganization but also development work to change routines or methods, that's what I thought of the last time we did something like that, and then I tried to remember and could not think of anything, it's sort of intertwined, and I think that is difficult. (R11)
	Essence	(4)	When I think about it, I probably didn't look back more than two weeks, actually, so I did not think about the average for a whole month; in my case, then, I would have responded in the same way whether it was the last month or 14 days, maybe even the last week, actually. (R7)
	Direction	(5)	I actually think it's very good, because I think you should base things on all the things that are good, because then you still sort of get the negative things anyway. (R6)
		(6)	[…] the questions are phrased in a positive way, and that can of course be good so you don't phrase them in a negative way every other time, because then you may easily mark your response the wrong way as you are all of a sudden supposed to think in the opposite direction when responding, so I guess it's good that it goes in the same direction. (R13)
		(7)	Well, it [positive and negative words] also becomes easier, you know, than if they used only good or bad because then there are more alternatives […] (R8)
	Keywords	(8)	[…]to get negative feedback does not mean it has to be negative, maybe negative feedback can be about something that is not working so well, and then you need to know what it is so you can do something about it; mostly, I think feedback is good as it is positive regardless. (R2)
		(9)	I guess I think it [personal challenge] is positive; it depends whether it becomes too big so it becomes a burden, but it can be kind of fun to have a challenge. (R8)
Responding	Strategy	(10)	I do not think of any change process in particular and had answered in general, but if I had been juggling with others then it might had been like, oh yes that's right, and we had thought about any change. (R1)
		(11)	Well, first of all I read the entire first section, all the questions, as I guess I feel that they are part of its own section; then I start with the first question again and think about it more closely, well, how do I feel when I get support and help in my work; totally agree, and maybe here I will respond like this, and there I may need to think about it a bit more. (R9)
		(12)	I thought about both how I feel sort of about life in general; then I thought about this one that it has to do with my working capacity and what I think about my job regarding all these questions, so I thought about it from different aspects. The prerequisite is that I feel quite okay so I can do my job. If I don't, my job would be disastrous. This is sort of what you want to find out; the employer is interested in how I feel, so I can function in my job, and that relates to my entire being. (R9)
		(13)	I still sort of feel that one has to learn somehow and complete the surveys; otherwise you cannot come back later and say something; you still want to be in on the discussions afterwards, but sometimes it is hard to know exactly how to respond; it is easy to respond to some of the questions but sometimes it can be a bit difficult. (R12)
	Alternatives	(14)	The opposing words are sort of the extremes, and it does not feel strange to evaluate myself between these, then it's my own personal values on all these health questions. I understand them. (R9)

Table 3. An overview (from left to right) of how different quality aspects relate to the study results, various questionnaire and respondent factors, and respondent actions.

Quality aspects	Questionnaire and respondent factors^a	Categories and subcategories from the study results	Tourangeau´s respondent actions^b
Face validity	Questionnaire factor	Interpreting	Comprehension
	Cognitive burden	Difficulty Essence	Literal Intended
Content validity	Questionnaire factor	Interpreting	Comprehension
	Content relevance Item relevance	Direction Keywords	IntendedJudgement
User validity	Questionnaire factor	Responding	Retrieval
	Time burden	Strategy Alternative	Factual response strategy
			Judgement
			Response
			Editing Alternatives
Reliability	Respondent factor	Strategy	Retrieval
	Attitude to surveys Decision to participate		Attitudinal response strategy

^a Wenemark et al. (2010).

^b Tourangeau (1984), Tourangeau et al. (2000).

Table 4. Strengths and weaknesses from the evaluation of the Salutogenic Health Indicator Scale (SHIS) and the Work Experience Measurement Scale (WEMS), and suggestions made for the improvement of identified weaknesses.

Strengths	Weaknesses	Suggestions for improvement of identified weaknesses in SHIS and WEMS
Low cognitive burden Keyword comprehension is similar Low time burden Response alternatives fit Content is easily related to the daily work	In WEMS, there are five statements that are medium or hard to understand	Clarifications of instructions are made. One example in WEMS is the statements about autonomy, where the instruction is enriched with a help text in brackets: Please enter your opinion on the following statements about your work situation. (Supply your entries based on your position or your work tasks.).
	Interpretation of the word workplace differs for respondents working at more than one workplace	In WEMS, the word workplace is changed to work
	Respondents are feeling obliged to complete the questionnaires at work	Talk about attitudes, usefulness and meaningfulness of surveys in survey processes

Wenemark, M., Hollman Frisman, G., Svensson, T., & Kristenson, M. (2010). Respondent satisfaction and respondent burden among differently motivated participants in a health-related survey. Field Methods, 22(4), 378–390.

 Web of Science ®Google Scholar

Tourangeau, R. (1984). Cognitive sciences and survey methods. In T. Jabine, M. Straf, J. Tanur, & R. Tourangeau. (Eds.), Cognitive aspects of survey methodology: Building a bridge between the disciplines (pp. 73–100). Washington, DC: National Academy Press.

 Google Scholar

Tourangeau, R., Rips, L., & Rasinski, K. (2000). The psychology of survey response. New York, NY: Cambridge University Press.

 Google Scholar