Validation of Health Behavior and Stages of Change Questionnaire

Abstract

Background

The transtheoretical model (TTM) has been widely used to promote healthy behaviors in different groups. However, a questionnaire has not yet been developed to evaluate the health behaviors that medical practitioners often consider in individuals with cancer or at a high risk of developing cancer.

Purpose

The aim of this study was to construct and validate the Health Behavior and Stages of Change Questionnaire (HBSCQ), which is based on the TTM and health recommendations related to risk and factors that protect against cancer.

Methods

Content validity was conducted in two phases (qualitative and quantitative). Item difficulty index, item discrimination index, and discrimination coefficient were obtained based on the classical test theory. Finally, Cronbach’s alpha was used.

Results

Measure of concordance showed scores considered adequate and excellent. The item discrimination index obtained a rating of “excellent” and suggested the preservation of all items. The discrimination coefficient scores are >0.74. The global internal consistency of the HBSCQ was 0.384. HBSCQ specification between groups of internal consistency for the sample of men was 0.712 and that for the sample of women was 0.378.

Conclusion/implications for practice

The HBSCQ represents a proposal for a fast, simple, and innovative screening test, which aims to identify persons who may benefit from interventions to promote health behaviors delimited to the stage of change.

Keywords:

• screening tests
• health promotion
• lifestyle
• transtheoretical model

Introduction

Prevention of chronic diseases has become one of the main challenges of the health system in Mexico.¹ Health behaviors are a key component for reducing the risk of diseases such as diabetes, heart disease, and cancer,² which together account for 68% of the causes of death worldwide.³

The transtheoretical model (TTM) has been used for behavior change. It considers that individuals pass through stages of change, characterized by different attitudes and motivations. These stages are pre-contemplation, contemplation, preparation, action, maintenance, relapse, and termination.⁴ The model has been used widely, showing its effectiveness in behaviors such as quitting smoking,^5,6 alcohol intake,⁷ eating habits,⁸ physical activity,^9,10 and use of screening tests.^11,12 To evaluate the stages of change, these studies are supported by criteria constructed from proposed definitions,^6,11,12 self-report questionnaires,^9,13,14 or evaluating various groups of behaviours.¹⁵

The Health Behavior and Stages of Change Questionnaire (HBSCQ) is based on the TTM and was designed to obtain a rapid assessment of risk and health behaviors and the stage of change for participants of genetic counseling for cancer. This population is key to prevention and promotion of health behaviors because they have a higher risk of developing cancer or relapse, because of clinical characteristics, family history of the disease, or a genetic mutation.¹⁶ However, the questionnaire can be applied to any segment of the population.

An extensive literature review was performed, and it was found that there is no questionnaire that fits the stages of change to the behaviors that authors needed. The questionnaires and criteria that have been used to evaluate smoking, alcoholism, and physical activity from the TTM are applied to diverse populations. Furthermore, the approach of studying each behavior separately predominates.^6,9,13,14 Few studies have associated health behaviors, TTM, and the use of screening tests for cancer prevention.^11,12 Other studies that are aimed at people with cancer or the screening of the disease evaluate health behaviors through criteria without a theoretical model,^17,18 through questionnaires not intended for cancer patients,¹⁹ or by using a different model than TTM.²⁰

The construction and validation of factors to recognize the theoretical and conceptual assumptions outlined earlier were necessary. It was decided to prepare an easy and brief questionnaire that guides the health professional to make decisions about effective intervention to promote health behaviors. This study aimed to construct and validate the HBSCQ.

Methods

Subjects

In total, 454 questionnaires were applied, of which 13 were eliminated because they were not fully completed. Of the 441 questionnaires analyzed, 46 were answered by participants of genetic counseling for cancer or cancer patients, 184 by university students, and 211 by the general population.

In the process of content validity, five experts in the TTM and health behaviors were invited to participate in the first step (qualitative analysis) and four in the second step (quantitative analysis).

Questionnaire

The HBSCQ is a new proposal and was constructed considering the postulates of TTM and was based on “the criteria for the adoption of mammographic behavior” defined by Andreu et al.¹² This structure was replicated for five sections, considering national and international recommendations for each behavior:^21,22 smoking, alcohol consumption, nutritional consultation, physical activity, and use of mammography and/or ultrasound. It has seven response options for alcohol consumption, nutritional consultation, and physical activity corresponding to each of the stages of change. In smoking, there are eight options for answers, considering two for the termination stage. However, the options for smoking were computed with the same code at the time of analysis. Finally, the use of mammography and/or ultrasound has six response options, eliminating the termination stage due to the recommendation of an expert in health promotion, arguing that the processes for screening are continuous, age-adjusted, dependent on the findings, and so on. The last section was answered only by women. The questionnaire also collects some demographic data. The HBSCQ requests in each section that the participants select the option that most reflects what he or she does (Table 1).

Table 1 Health Behaviors and Stages of Change Questionnaire

Instructions: Below is a group of phrases related to different behaviors. Please select the letter of the clause that most closely matches what you do.

Smoking

a. I smoke now, and I have not thought about quitting.

b. I currently smoke but I’m considering quitting.

c. I want to quit and I have begun to reduce the number of cigarettes that I smoke.

d. I quit smoking less than 6 months ago.

e. I have not smoked for more than 6 months.

f. In recent years, I’ve stopped smoking for more than 6 months but smoke occasionally.

g. I quit smoking several years ago, and I don’t feel any desire to smoke again.

h. I have never smoked.

Alcohol consumption

a. I drink 7 or more alcoholic drinks per week (can or bottle of beer, a glass of wine, etc), and I have not thought about changing this habit.

b. I drink 7 or more alcoholic drinks a week, but I’ve thought about reducing my consumption.

c. I want to decrease my alcoholic consumption (7 or more drinks per week), and I have thought of ways that can help me achieve this.

d. Recently (less than 6 months), I have decreased my alcohol intake to less than 7 drinks per week.

e. More than 6 months ago I changed or have diminished my alcohol intake to less than 7 drinks per week.

f. I take less than 7 drinks of alcohol per week, but sometimes I drink more.

g. I have never drunk alcohol or I have not drunk alcohol for several years.

Nutritional consultation

a. I have been to see a nutritionist and although I have already been discharged I continue the nutritional recommendations I received and I have incorporated them into my daily life.

b. I have never been to a nutritional consultation, and I have not thought to go.

c. I have never been to a nutritional consultation, but I’m thinking of going, within the next 6 months.

d. I have a first appointment with the nutritionist.

e. I went to the nutritionist in the last 6 months, and I have followed his/her recommendations.

f. I attend my consultations with a nutritionist and I have followed his/her recommendations for more than 6 months.

g. I go to the nutritionist, but I usually don’t follow his/her recommendations, or I miss my appointments.

h. I go to the nutritionist, and although I have been discharged, I follow the nutritional recommendations that I received, and I have incorporated them into my daily routine.

Physical activity

a. I tend to take part in some sport or recreational physical activity regularly (at least 3 times per week, 30 minutes per day), but sometimes I do not.

b. I have been playing some sport or participating in some kind of recreational physical activity regularly (at least 3 times per week, 30 minutes per day), it is part of my daily life.

c. I don’t do any sport or recreational physical activity routinely (at least 3 times a week, 30 minutes a day), such as going for a walk or run; and I have not thought about starting any of these activities.

d. I don’t do any sport or recreational physical activity routinely, but I do think about starting to do some exercise.

e. I am planning to begin exercising, or have already started, but still cannot do it regularly.

f. In the last 6 months I started practicing a sport or recreational physical activity routinely.

g. More than 6 months ago I started practicing a sport or recreational physical activity routinely.

h. I do recreational physical activity or some sport regularly, but sometimes I cannot do it 3 times a week or more.

i. For several years I have played sports or taken part in recreational physical activity routinely, it is part of my lifestyle.

Use of mammography and/or ultrasound

a. I have never had a mammogram/breast ultrasound and I don’t plan on ever having either test done.

b. I have never had a mammogram/breast ultrasound, but I plan on doing one or both in the next 6 months.

c. I have an appointment for my first mammogram or breast ultrasound.

d. I had my first mammogram/breast ultrasound recently and I intend to have one done at least every 2 years.

e. I have had mammograms/breast ultrasounds for several years routinely (at least every two years)

f. I’ve done some mammography/breast ultrasound tests, but I do not do it routinely (at least every 2 years)

Note: The original version of HSBCQ was applied in Spanish.

Procedure

Content validity

The HBSCQ had a rigorous validation process. Content validity was determined in two steps: first, with a qualitative analysis. An evaluation form was developed, and five experts in the TTM and health behaviors were invited to participate. They corrected the writing style and answered the question “How good is this item to satisfy the objective of this area?” on a scale of 0–4, ranging from “Not good” to “Very good.” The HBSCQ was modified based on the recommendations of the experts, following one or more of the following criteria: 1) the modification added clarity or understanding to the sentence; 2) the score given to the item by two or more experts was <3; 3) an expert provided scientific or relevant evidence about regulations or health recommendations for modification or removal of any sentence.

With the corrected HBSCQ, the second step of content validity was quantitative analysis. Four experts, two of them were new, were invited, and they evaluated a new format following two criteria. First, with the individuals presented at random, they indicated the number corresponding to the stage that they considered reflected the phrase: 1) pre-contemplation, 2) contemplation, 3) preparation, 4) action, 5) maintenance, 6) relapse, and 7) termination. Second, they chose “yes” when they considered that the sentence was appropriate to reflect the stage of change in which the person finds themselves with regard to health behavior and marked “No” in the opposite case. The Kappa statistic was applied. Values between 0.40 and 0.59 reflect an acceptable agreement, between 0.60 and 0.75 an appropriate agreement, and >0.75 reflects an excellent agreement.

Item validity

After content validity, piloting of HBSCQ was performed. The HBSCQ was evaluated under the classical test theory. The item difficulty index shows that the higher the value for difficulty, the more people practice risky behavior. The item discrimination index distinguishes people with higher and lower practice of risky behaviors. Values >0.39 represent an excellent level of discrimination. The coefficient of discrimination measures the level of overall risk. For the above analyses, the health behaviors of individuals in the action, maintenance, and termination stages were considered, and the remaining stages are considered to have a risk factor.

Finally, to determine the internal consistency, Cronbach’s alpha was used. Statistical analyses were performed by using SPSS v.20 and Microsoft Excel 2013.

Ethics

The application of the questionnaire was approved by the institutional review board at the University of Guadalajara with the registration number C.I. 035-2011. According to the article 11.3 of the Norma Oficial Mexicana NOM-012-SSA3-2012, which establishes the criteria for the execution of health research projects in humans, in cases of investigations without risk or minimal risk, the informed consent will not be required. The purpose of the study was verbally explained to all participants, which only consisted of completing the HBSCQ. All participants gave their verbal consent, and they were asked to fill in the personal information requested and complete the questionnaire themselves, with the freedom to stop or withdraw at any time. The survey did not take more than 5 min. It was done in genetic counseling clinics, in public places, and with some students inside the University of Guadalajara.

Results

Kappa as a measure of concordance showed scores >0.66 (Table 2). Analyzing for each of the health behaviors (Table 3), the highest concordance between judges was in the area of smoking (scores >0.83). In the section of physical activity, it has been noted that one judge rates an acceptable level of concordance (0.50), in contrast to the rest of the judges who obtain the maximum score (1).

Table 2 Analysis of concordance between judges on the stage of change that expresses each sentence in the HBSCQ

	Judge 1	Judge 2	Judge 3	Judge 4
Judge 1		0.69	0.72	0.66
Judge 2	0.69		0.83	0.90
Judge 3	0.72	0.83		0.86
Judge 4	0.66	0.90	0.86

Note: For all analyses, p<0.001.

Abbreviation: HBSCQ, Health Behaviors and Stages of Change Questionnaire.

Table 3 Analysis of concordance between judges on the stage of change that each sentence expresses by health behavior

Judge	Judge 2					Judge 3					Judge 4
	1	2	3	4	5	1	2	3	4	5	1	2	3	4	5
1	0.83	0.66	0.67	0.5	0.8	1	0.83	0.83	0.5	0.45	0.83	0.67	0.67	0.5	0.62
2						0.83	0.83	0.83	1	0.61	1	0.67	1	1	0.81
3											0.83	0.83	0.83	1	0.80

Notes: 1. Smoking, 2. Alcohol consumption, 3. Nutritional consultation, 4. Physical activity, 5. Use of mammography and/or ultrasound. For all analysis, p<0.001.

Regarding the second criterion for evaluating whether the sentence was appropriate to reflect the stage of change that the person records in relation to each health behavior, judges 1 and 2 answered “Yes” in all cases, whereas judges 3 and 4 did so in only 30 of the 34 sentences (88.2%).

In relation to the pilot, the characteristics of the sample are presented in Table 4. The results of the item difficulty index show low scores on smoking (0.22) and alcohol consumption (0.17). Conversely, nutritional consultation (0.78), physical activity (0.72), and use of mammography and/or ultrasound (0.69) had high scores. Regarding the item discrimination index, all items obtained a rating of “excellent” and suggested preservation (Table 5). The discrimination coefficient scores are >0.74.

Table 4 Measures of central tendency, dispersion, frequency, and percentages of the characteristics of the sample

	M±SD	n (%)*
Age (years)	3±1
Total risk behaviors per participant	2.38 (1.06)
Age group
<30 years		224 (50.8)
30–39 years		74 (16.8)
40–49 years		86 (19.5)
≥50 years		56 (12.7)
Gender
Female		306 (69.2)
Male		135 (30.8)
Years of study
0 years		3 (0.7)
<6 years		31 (7)
6–9 years		65 (14.7)
10–12 years		106 (24)
13–17 years		209 (47.4)
>17 years		25 (5.7)
Residence
Metropolitan area of Guadalajara		305 (69.2)
Other		136 (25.5)
Occupation
Employee		177 (40.1)
Unemployed		259 (58.7)
Cancer diagnosis
Yes		44 (10)
No		397 (90)
Diagnosis of another disease
Yes		71 (16.1)
No		379 (83.9)

Note:

* Frequency (percentages).

Abbreviations: M, mean; SD, standard deviation.

Table 5 Scores of item validity

	Item difficulty index	Item discrimination index	Discrimination coefficient
Smoking	0.22	0.85	0.81
Alcohol consumption	0.17	0.95	0.74
Nutritional consultation	0.78	0.48	0.79
Physical activity	0.72	0.51	0.87
Use of mammography and/or ultrasound	0.69	0.56	0.89

The internal consistency of the HBSCQ Cronbach’s alpha was 0.384. Eliminating the variable using mammography and/or ultrasound (because it was only applied to women in the sample), reliability was 0.427. Making HBSCQ specification between groups of internal consistency for the sample of men was 0.712 and for the sample of women, it was 0.378.

Conclusion/implications for practice

The aim of this study was to describe the construction process, piloting, and validation of the HBSCQ. Some authors have generated questionnaires^9,13 or established criteria^6,11,12 for the location of subjects in each of the stages of change. However, there was no questionnaire that would integrate the five health behaviors that the HBSCQ proposed. Besides, most of the existing questionnaires were generated in other countries; Sarbandi et al¹⁴ recommend that care should be taken for the cultural adaptation of questionnaires, for example, local regulations regarding use of screening tests such as the use of mammography or/and ultrasound.

The qualitative study of content validity with the help of experts provided clarity and checked the appropriateness of each item in the questionnaire regarding the particular theoretical framework. The HBSCQ presented was globally adequate (50%) and proved excellent concordance (50%) among judges. The same trend was observed while analyzing the concordance for each of the health behaviors, thus ensuring content validity aimed at both the TTM and the recommendations of health in Mexico. The questionnaire offers health professionals in Mexico the option to apply a fast, simple, and innovative screening test, to identify persons who may benefit from interventions to promote health behaviors, and at the same time they can count on intervention strategies delimited to the stage of change.^6,8–11,23

As for item validation, low alcohol consumption and smoking scores reflect that most participants evaluated did not have these risk behaviors in contrast to the scores in the areas of nutritional consultation, physical activity, and use of mammography and/or ultrasound.

Concerning the internal consistency of HBSCQ, scores >0.6 were expected. As this is the first effort of validation of HBSCQ, favorable results of the content and item validity, in addition to the peculiarity of the sample and the object study, were expected. We aim to improve the HBSCQ to achieve acceptable standards. One way is to calculate different reliability coefficients for HBSCQ depending on the different sections or groups²⁴ and reduce, extend, or modify the questionnaire to obtain greater reliability. For example, in the HBSCQ, considering only the sections of smoking and alcohol consumption, internal consistency reached 0.643, and considering only the group of men, the internal consistency was 0.712.

The following steps will extend the application of HBSCQ in the population of interest (cancer patients undergoing genetic counseling), accompanied by the application of a clinical interview and/or questionnaires on social desirability, because we are interested to know whether this variable could be influencing the answers given by users. The aim of HBSCQ is to be a reliable method to identify individuals who may benefit from interventions based on the stage of change, using the strategies that the same model and other authors have proposed.^6,9,10,13 Future work must show the utility of HBSCQ. The application of HBSCQ for other vulnerable populations or groups of intervention is encouraged, and even cross-cultural adaptations are made, which will strengthen the effectiveness of TTM, using programs that target multiple risk factors, which is the reality of chronic diseases.²⁵

This study has several limitations. In particular, the sample was not homogeneous or random, which therefore affects the results of the HBSCQ.²⁴ The highest score in the variable difficulty index in the section of mammography and/or ultrasound use compared with physical activity may be altered due to the percentage of participants in genetic counseling or cancer patients included in the sample (~10%), having different behaviors to the general population. Although the purpose of HBSCQ piloting was its validation, not precisely for individuals receiving genetic counseling or cancer patients, our daily interaction with this population and the main interest in developing this questionnaire represent a bias in the selection of the sample, which should be rigorously taken care of in the following studies. A comparative analysis of the application of HBSCQ between the general population and cancer patients will be reported separately. We also intend to expand the sample of patients evaluated in genetic counseling (with and without a personal history of cancer) to contribute to greater scientific evidence among this specific group.

On the other hand, the number of items (five), sample characteristics, and the selection of behaviors that comprise the questionnaire could be influencing the low scores in reliability analysis. A detailed analysis should be performed on HBSCQ to achieve optimal internal consistency.

Another limitation of HBSCQ is that it only evaluates the use of mammography and/or ultrasound, discarding other types of cancer screening tests such as colonoscopy and without distinguishing age group. These considerations may be included in an improved version of HBSCQ, or in adaptations made by other professionals who may be interested not only in cancer but other diseases.

Furthermore, in the nutritional consultation section, the factors were characteristic of Mexican institutions, where nutritional consultation is provided free of charge to patients. However, outside this context, it ignores other ways in which people can adhere to a healthy diet, such as through self-learning (books, the Internet, television, and magazines), especially in those groups that find it harder to access the services of a nutritionist. In addition, the HBSCQ omits the possibility of the person following recommendations from other health professionals such as general practitioners.

Finally, other measures should be considered to accompany the HBSCQ, or it should be contrasted with other questionnaires to provide external evidence on the validity of the questionnaire.

Disclosure

The authors report no conflicts of interest in this work.

Acknowledgments

The authors thank the National Council of Science and Technology (CONACYT) for the support received through the Inter-Institutional Doctorate in Psychology. The authors wish to thank the Directorate for Research Support and Postgraduate Programs at the University of Guanajuato for their support in the editing of the English-language version of this article.