The reliability and validity of a Chinese-version Short Health Anxiety Inventory: an investigation of university students

Abstract

Background

The Short Health Anxiety Inventory (SHAI) is widely used in English-speaking populations, with good reliability and validity. For further research needs in the Chinese population, it was translated into a Chinese version (CSHAI). Furthermore, the reliability, validity, and cutoff score were examined in a nonclinical population in the People’s Republic of China.

Methods

Three hundred and sixteen undergraduates were evaluated by a set of questionnaires including CSHAI, Zung Self-Rating Anxiety Scale (SAS), Zung Self-Rating Depression Scale (SDS), and the State-Trait Anxiety Inventory (STAI). Fifty-eight students completed CSHAI again after 30 days.

Results

The two-factor model had satisfactory fit indices. The correlation coefficients between each item with the CSHAI total and each subscale were between 0.386 and 0.779. The Cronbach’s alpha coefficients of CSHAI total and its subscales were 0.742, 0.743, and 0.788, respectively, and the split-half coefficients were 0.757, 0.788, and 0.912. The test–retest correlation coefficients were, respectively, 0.598 (P<0.001), 0.539 (P<0.001), and 0.691 (P<0.001). Convergent validities were respectively 0.389–0.453, 0.389–0.410, and 0.250–0.401, and discriminant validities were −5.689 (P<0.001), −5.614 (P<0.001), and −3.709 (P<0.001). The cutoff score was 15.

Conclusion

CSHAI showed good factor structure, reliability, convergent validity, and discriminant validity, and 15 was determined to be the appropriate cutoff score for screening health anxiety.

Keywords:

• health anxiety
• confirmatory factor analysis
• cutoff score

Introduction

Health anxiety (HA) refers to a negative interpretation and fears about the meaning of both ordinary and unusual bodily sensations.¹ The prevalence in the general population varies,^2–4 and it considerably reduces life quality and increases the chance of medical consultation and seeking of psychotherapeutic or psychiatric treatment.^5,6 Hypochondriasis is considered an extreme form of HA;⁷ however, HA and hypochondriasis are not distinguished clearly.^2–4 Actually, hypochondriasis and HA share a common component of phobia (and, more broadly, health- and disease-related concerns), but that does not seem to be the case with disease conviction, as definitions of HA usually do not include an idea or belief that a serious illness is present.⁸ This results in various measurements for HA, including the Illness Attitudes Scale (IAS), Whiteley Index (WI), Structured Diagnostic Interview for Hypochondriasis (SDIH),⁹ and Short Health Anxiety Inventory (SHAI).¹⁰

Salkovskis et al¹⁰ developed the Health Anxiety Inventory (HAI) (64 items) and a shortened version of this scale, the SHAI (18 items). The shortened version was sensitive to both normal levels of health concern and severe HA. In addition, SHAI was demonstrated to be an appropriate measurement that was sensitive to both mild and more severe forms of HA in both medical and nonmedical samples.¹¹ Adequate-to-excellent internal consistency in undergraduate students and strong construct validity was affirmed.¹² The factor structure,^13–15 reliability and validity, cutoff score, versions in different languages, and various populations have been examined. The original factorial structure of SHAI included a two-factor model^10,11,15 and a three-factor model.¹⁶ The results of the above investigations are inconsistent, varying with the number of items. Nevertheless, the two-factor structure of SHAI has received the greatest support and can provide a more comprehensive assessment of the factor structure of HA.

The English-version SHAI has been widely explored, mainly in English-speaking populations. Just one study, with a sample of 832 Spanish secondary school adolescents, used the Spanish version.¹⁷ The results indicated adequate reliability of the inventory and suggested SHAI may be considered an appropriate instrument for assessing HA in Spanish-speaking adolescents. In Asian countries, there is as yet no appropriate measurement for screening HA either in clinical samples or nonclinical populations. The Zung Self-Rating Anxiety Scale (SAS) is the most widely used measure in the People’s Republic of China for screening anxiety.^18–20 The main aim of this study was to analyze the reliability and validity of SHAI for its possible use in assessing HA in Chinese general populations. The second aim was to explore the cutoff score of the Chinese-version SHAI (CSHAI).

Methods

Participants

Three hundred and sixteen healthy medical students (aged from 18 to 27 years) participated in this study. There were 122 men (mean age 21.69 years with standard deviation [SD] 1.56, range 19 to 26) and 194 women (mean age 21.76 years with SD 1.72, range 18 to 27). There was no significant age difference between the two sex groups (t=−0.387; 95% confidence interval [CI]: −0.45–0.30; P=0.699). Another 61 students participated in the test–retest reliability research and they completed the CSHAI twice every 30 days. Finally, 17 men (mean age 22.88 years with SD 1.27, range 20 to 25) and 41 women (mean age 21.88 years with SD 1.52, range 20 to 27) were retained. There was a significant age difference between the two sex groups (t=2.397; 95% CI: 0.16–1.84; P=0.02) either. All participants were confirmed to have no history of serious illness (including mental disorders and neurological diseases).

Measures

The participants were asked to fill in the following four Chinese-version questionnaires.

CSHAI

The CSHAI¹⁰ has two factors, corresponding to 1) the feared likelihood of becoming ill (Illness Likelihood [IL], 14 items), and 2) the feared negative consequences of becoming ill (Negative Consequences [NC], four items). Each item of the CSHAI consists of four statements that range from “I do not” (0) to “I spend most of my time” (3). The total scores are from 0 to 54.

The Zung Self-Rating Anxiety Scale

Zung compiled the Self-Rating Anxiety Scale (SAS) in 1971, and it is a 20-item, self-report measure of anxious symptoms.²¹ Each of the items is ranked on a four-point Likert scale, ranging from “never occurring” or “a little of the time” to “most of the time”. Responses were summed to calculate a total score, with higher scores indicating greater levels of anxious symptomatology. Good validity has been demonstrated for the Chinese version of SAS.²² Standard scores above 50 suggest clinically significant levels of anxiety in a Chinese population.²²

The Zung Self-Rating Depression Scale

The Zung Self-Rating Depression Scale (SDS)²³ is a 20-item self-report tool which was developed to measure depressive symptoms and for depression screening. In a study of the Chinese-version SDS in students,²⁴ good internal consistency was confirmed, with a Pearson’s correlation coefficient of 0.313–0.640.

The State-Trait Anxiety Inventory

The State-Trait Anxiety Inventory (STAI)²⁵ is a 40-item measure of anxiety. It can measure both state anxiety (how anxious a person is feeling at a particular moment [S-AI]) and trait anxiety (how dispositionally anxious a person is across time and situations [T-AI]) and consists of two separate subscales containing 20 items each. Each item is scored from 1 to 4, with the total score ranging from 20 to 80 for each scale and high scores indicating increased anxiety. Good test–retest reliability has been demonstrated,²⁵ with a Pearson’s correlation coefficient of 0.73–0.77 in S-AI and 0.31–0.33 in T-AI.

Procedures

The SHAI was translated by two master’s students, and a physician proficient in English without access to the original English version performed back-translation. Then, a meeting was held to discuss each item’s suitability for a Chinese population. Finally, two psychiatrists checked the translated version and agreed upon the primary version of CSHAI. A pilot test of the Chinese-language survey was conducted with 30 participants. There were no reports of misunderstandings, so this version was used as the final version.

The participants in this study were adult volunteer university students. Three hundred and sixteen students filled out the paper-based questionnaires anonymously over 2 days, and the entire procedure took approximately 20–30 minutes. Fifty-eight students completed the CSHAI twice, with an interval of 30 days before the second instance. The students did not receive an academic or other reward for participation. The study procedure was approved by the ethical committee of Zhongda Hospital, which is affiliated to Southeast University (Nanjing, People’s Republic of China).

Analyses

To complete the analyses, the Predictive Analytics Software (PASW) Statistics 18 package and IBM SPSS Amos 22 were used (IBM Corporation, Armonk, NY, USA). The factor structure of the CSHAI was confirmed following the Bentler and Bonett²⁶ criteria, using three commonly used indices: comparative fit index (CFI), root mean-square error of approximation (RMSEA), and Satorra-Bentler chi-square. The value of CFI should exceed a recommended cutoff value of 0.90 (more liberal) or 0.95 (more strict), and a value of RMSEA less than 0.08 (more liberal) or 0.05 (more strict) indicates a good fit.²⁷ Internal consistency was assessed with the Cronbach’s alpha coefficient and split-half coefficient. Convergent validity was documented using a Pearson’s correlation coefficient by comparing the CSHAI total with the SAS. Comparison between the anxiety group and non-anxiety group was analyzed by independent-samples t-test. P-values less than 0.05 were considered to indicate statistical significance.

The cutoff score was determined by the Youden index, combining the sensitivity value and specificity value²⁸ that resulted from the receiver operating characteristic (ROC) curve. The areas of ROC curve could be used as an index to examine the precision of the test. The Youden index was calculated by the following formula: $\text{Youden index}=\text{sensitivity value}+\text{specificity value}-1$(1)

The maximum of the Youden index is the best cutoff value. To verify the accuracy and effectiveness of this cutoff value, we calculated the accordance rate compared with SAS and used multiple linear regression to observe the comparison directly.

Results

Confirmatory factor analysis

Standardized loadings are shown in Table 1 and the confirmatory factor analysis model in Figure S1. All item loadings were high, with one exception: item 10 for IL. Indices for the original two factors, χ² (134) =274.282, P<0.001, CFI =0.901, and RMSEA =0.058, indicated a good fit.

Table 1 Confirmatory factor analysis: factor loadings (N=316)

SHAI item	ILa (items 1–14)	NCa (items 15–18)
1	0.581
2	0.345
3	0.348
4	0.616
5	0.762
6	0.655
7	0.590
8	0.475
9	0.556
10	0.277
11	0.663
12	0.677
13	0.368
14	0.355
15		0.608
16		0.586
17		0.701
18		0.556

Note:

a A subscale of CSHAI.

Abbreviations: CSHAI, Chinese-version Short Health Anxiety Inventory; IL, Illness Likelihood; NC, Negative Consequences.

Internal consistency

Table S1 shows correlations of the CSHAI total with IL, NC, and each item, as determined by Pearson’s correlation coefficient. The coefficient of determination ranged from 0.392 to 0.700 for each item (P<0.01), 0.965 for IL (P<0.01), and 0.731 for NC (P<0.01). In addition, the correlation coefficient of IL with items 1 to 14 ranged from 0.417 to 0.730 (P<0.01). Moreover, NC with items 15 to 18 ranged from 0.651 to 0.780 (P<0.01). The remarkably high coefficient between CSHAI and each item indicated the high consistency of CSHAI. The correlation between both factors was identified as moderate (0.526), indicating that they are related but measure different aspects of HA.

The analysis of the internal consistency of the CSHAI total generated a Cronbach’s alpha coefficient of 0.742. The coefficients for the IL and NC subscales were 0.743 and 0.788. The split-half coefficients of the CSHAI total, IL, and NC were 0.757, 0.788, and 0.912, respectively.

Test–retest reliability

For the 58 participants who completed the CSHAI twice, Pearson’s correlation coefficient was 0.560 for CSHAI total (P<0.01), 0.438 for IL (P<0.01), and 0.720 (P<0.01) for NC, indicating a relatively satisfactory level of test–retest reliability.

Convergent validity

The correlations of CSHAI with SAS, SDS, S-AI, and T-AI are presented in Table 2. CSHAI total was significantly correlated with SAS (r=0.390, P<0.01), S-AI (r=0.429, P<0.01), and T-AI (r=0.454, P<0.01). On the contrary, the correlation of CSHAI with SDS was not significant (r=0.078, P>0.05). Similar to CSHAI total, IL and NC were also significantly correlated with SAS, S-AI, and T-AI.

Table 2 Correlations of SHAI with other scales

	SAS	SDS	S-AI	T-AI
SHAI total	0.389**	0.077	0.428**	0.453**
ILa	0.389**	0.086	0.391**	0.410**
NCa	0.250**	0.027	0.369**	0.401**

Notes: Data presented as Pearson’s correlation coefficient (r).

a A subscale of SHAI, S-AI and T-AI are subscales of STAI.

** P<0.001.

Abbreviations: SHAI, Short Health Anxiety Inventory; IL, Illness Likelihood; NC, Negative Consequences; SAS, Zung Self-Rating Anxiety Scale; SDS, Self-Rating Depression Scale; S-AI, state anxiety; T-AI, trait anxiety; STAI, State-Trait Anxiety Inventory.

Discriminant validity

Three hundred and sixteen participants were divided into two groups according to the cutoff score of 50 SAS standard scores. There were 39 students in the anxiety group (accounting for 12.34%) and 277 in the non-anxiety group (accounting for 87.66%) (see Table 3). Comparing CSHAI scores, we found that there were significant differences between the anxiety and non-anxiety group.

Table 3 Comparison of CSHAI scores between students in the anxiety group and non-anxiety group

	Non-anxiety group (N=277)	Anxiety group (N=39)	t
SHAI total	11.28±5.43	16.97±8.27	−5.689**
ILa	9.03±4.37	13.54±6.62	−5.614**
NCa	2.26±1.77	3.44±2.35	−3.709**

Notes:

a Data presented as mean ± standard deviation. A subscale of CSHAI.

** P<0.001.

Abbreviations: CSHAI, Chinese-version Short Health Anxiety Inventory; IL, Illness Likelihood; NC, Negative Consequences.

Cutoff value of CSHAI

The cutoff value of CSHAI was determined to be 15 by the ROC curve. Figure 1 demonstrates the ROC curve of CSHAI and SAS with the data of 316 students. The short dotted line is the curve of SAS and the long dotted line is the curve of CSHAI. SAS had a larger area under the curve than CSHAI, but the values were close. The area under the curve of CSHAI was 0.745 (P<0.001; 95% CI: 0.657–0.834) and that under SAS was 0.993 (P<0.001; 95% CI: 0–1). The results suggest that CSHAI had relatively good diagnostic accuracy.

Figure 1 ROC curves for CSHAI total and SAS.

Notes: The areas under the ROC curve for SHAI and SAS were 0.745 (P<0.001; 95% CI: 0.657–0.834) and 0.993 (P<0.001; 95% CI: 0–1), respectively.
Abbreviations: CI, confidence interval; ROC, receiver operating characteristic; SAS, Zung Self-Rating Anxiety Scale; CSHAI, Chinese-version Short Health Anxiety Inventory.

One-to-one correspondence and the average method were used while setting the cutoff value of CSHAI. Table 4 shows the SAS cutoff value of 49 corresponded to the maximum Youden index of 0.993. The maximum Youden index of CSHAI was 0.428 confirmed the cutoff value of CSHAI. Because of integral scores, the students with 15 or more in CSHAI total were considered to have HA. In this way, 92 students were verified to have HA, which accounted for 29.11%. This rate was significantly different to that of the anxiety group assessed by SAS (χ²=27.05, P<0.001).

Table 4 Cutoff values of SHAI and SAS

Cutoff value	CSHAI			Cutoff value	SAS
	Sensitivity	Specificity	Youden index		Sensitivity	Specificity	Youden index
−1.00	1.000	0.000	0.000	24.00	1.000	0.000	0.000
1.00	1.000	0.014	0.014	25.50	1.000	0.018	0.018
2.50	1.000	0.036	0.036	26.50	1.000	0.032	0.032
3.50	0.974	0.054	0.029	27.50	1.000	0.054	0.054
4.50	0.974	0.072	0.047	29.00	1.000	0.105	0.105
5.50	0.949	0.119	0.068	30.50	1.000	0.177	0.177
6.50	0.923	0.162	0.086	31.50	1.000	0.235	0.235
7.50	0.897	0.224	0.121	32.50	1.000	0.300	0.300
8.50	0.872	0.318	0.189	34.00	1.000	0.361	0.361
9.50	0.872	0.408	0.280	35.50	1.000	0.433	0.433
10.50	0.821	0.502	0.322	36.50	1.000	0.509	0.509
11.50	0.795	0.570	0.365	37.50	1.000	0.570	0.570
12.50	0.769	0.657	0.426	39.00	1.000	0.661	0.661
13.50	0.692	0.693	0.385	40.50	1.000	0.715	0.715
14.50	0.667	0.762	0.428	41.50	1.000	0.776	0.776
15.50	0.590	0.812	0.402	42.50	1.000	0.819	0.819
16.50	0.538	0.852	0.390	44.00	1.000	0.874	0.874
17.50	0.487	0.884	0.372	45.50	1.000	0.921	0.921
18.50	0.359	0.895	0.254	46.50	1.000	0.957	0.957
19.50	0.256	0.917	0.173	47.50	1.000	0.978	0.978
20.50	0.231	0.942	0.173	49.00	1.000	0.993	0.993
21.50	0.231	0.949	0.180	50.50	0.872	0.993	0.865
22.50	0.128	0.968	0.096	51.50	0.795	0.993	0.788
23.50	0.103	0.975	0.077	52.50	0.615	0.993	0.608
24.50	0.103	0.978	0.081	54.00	0.462	0.993	0.454
26.00	0.026	0.982	0.008	55.50	0.410	0.993	0.403
28.00	0.026	0.989	0.015	56.50	0.333	0.993	0.326
29.50	0.026	0.993	0.018	57.50	0.256	0.993	0.249
30.50	0.026	0.996	0.022	59.00	0.179	0.993	0.172
42.50	0.026	1.000	0.026	60.50	0.128	0.993	0.121
55.00	0.000	1.000	0.000	62.00	0.077	0.996	0.073
–	–	–	–	64.00	0.051	0.996	0.048
–	–	–	–	67.50	0.000	0.996	−0.004
–	–	–	–	71.00	0.000	1.000	0.000

Abbreviations: CSHAI, Chinese-version Short Health Anxiety Inventory; SAS, Zung Self-Rating Anxiety Scale.

Discussion

The aim of the present study was twofold: first, to confirm the reliability and validity of the two-structure CSHAI; and, second, to analyze the cutoff score of CSHAI in students. The confirmatory factor analysis showed satisfactory fit indices, confirming that, in the students from 18 to 27 years old, the CSHAI has the same two factors as those reported by Salkovskis et al,¹⁰ IL and NC. Moreover, it was close to the original version, with CFI =0.96 and RMSEA =0.052. In addition, CSHAI showed relatively good reliability and validity, based on recommendations that a Cronbach’s alpha coefficient over 0.80 is essential for acceptability as a basic research tool.²⁹ However, the alpha coefficient of the original CSHAI was 0.71 when used in a nonpatient sample, which was similar to NC, having an alpha coefficient of 0.72.¹⁰ This relates to the confusion of HA and hypochondriasis. HA may be a kind of symptom, but hypochondriasis is a kind of mental disorder. For nonpatients, HA screening and diagnosis is more difficult. This phenomenon is consistent with the Spanish-version CSHAI, which had low internal consistency of NC.¹⁷ In addition, the number of items influences results. Karademas et al’s investigation of students in 2008 used a 14-item SHAI,³⁰ and Boston and Merrick³¹ investigated community adults with an 18-item SHAI. However, in patients 14-item³² and 18-item^33,34 model all be used. Thus, the use of NC has some controversy when screening different populations.

In this study, SAS as a measurement to screen anxiety was selected to confirm the validity of CSHAI. In a paper by Rachman,³⁵ HA disorder as a new kind of anxiety disorder was associated with posttraumatic stress disorder, obsessive-compulsive disorder, panic disorder, and general anxiety disorder. The validity results suggested CSHAI total and the two subscales were significantly correlated with SAS, and two student groups were significantly different in CSHAI score.

The cutoff score was calculated using the Youden index, combining the sensitivity value and specificity value that resulted from the ROC curve in a sample of 316 students. The results showed 15 was the cutoff score for diagnosing HA in students. The area under the curve was relatively accurate for diagnosing HA, though it was below perfectly accurate (area under the curve =1).³⁶ This finding was consistent with Tang et al’s³⁷ study, which shows a cutoff point of 18 or higher in the SHAI reliably identifies people meeting diagnostic criteria for hypochondriasis, whilst a score between 15 and 17 represents a high level of HA but not enough to meet the diagnosis criteria of hypochondriasis. The study of Alberts et al¹² suggests a cutoff of 27 would apply, while Sulkowski et al³⁸ report a cutoff score as high as 38.

The controversies of cutoff value have brought some troubles to studies with CSHAI. Rachman³⁵ points out that severe HA, the extreme end of the continuum of HA, is often termed “hypochondriasis”. Although patients with HA or hypochondriasis would have similar avoidance and safety behaviors (such as avoiding going to hospital, repeated medical consultations and tests, self-checking), beliefs differ to some extent.^39–41 Hypochondriacal beliefs are resistant to disconfirmation. Unlike HA, in which future dangers are anticipated, in hypochondriasis, the danger is present and active, and the belief is fixed.³⁵ The dimensional characteristics and concept confusion of these two disorders closely relate to the construct and cutoff value of CSHAI. Therefore, verifying the reliability and validity of different versions of SHAI before investigation is necessary. This study enriches the usage of CSHAI, which represents one more measure for assessing HA and, at the same time, helps physicians to discover HA faster and more conveniently.

Limitations

In this study, the participants were recruited from only one university. Previous studies select particular populations as the subjects, which would be not referenced in a general population HA study. Future study must be conducted in both general and clinical populations. Moreover, comparisons between the two-factor structure (contain IL and NC) and only IL structure of SHAI were not explored. A wider-ranging study on a larger randomized population sample should be planned for further validation in a general Chinese population. Despite these limitations, the CSHAI was demonstrated to be useful in Chinese university students and had significant correlations with scales for screening anxiety.

Conclusion

The goal of this study was to validate CSHAI and construct a valid and reliable tool to measure HA in a Chinese population. This research confirms that CSHAI presented good internal consistency, highly satisfactory convergence, discriminant validity, and 15 as an appropriate cutoff score. It is promising for helping assessment of HA in the People’s Republic of China and will enrich the pools of SHAI study.

Acknowledgments

We would like to thank the medical students for completion of the questionnaires.

Supplementary materials

Figure S1 Interrelationship between the two facets of health anxiety.

Notes: IL and NC are subscales of SHAI. SHAI1 to SHAI18 are object properties which represent each item; e1 to e18 are residual variables corresponding to each object properties.

Abbreviations: SHAI, Short Health Anxiety Inventory; IL, Illness Likelihood; NC, Negative Consequences.

Table S1 Correlations of Chinese-version SHAI total with IL, NC, and each item

Item	SHAI total	ILa	NCa
ILa	0.965**
NCa	0.731**	0.526**
1	0.590**	0.607**
2	0.438**	0.460**
3	0.417**	0.487**
4	0.622**	0.638**
5	0.700**	0.730**
6	0.619**	0.640**
7	0.597**	0.610**
8	0.543**	0.570**
9	0.564**	0.581**
10	0.392**	0.417**
11	0.679**	0.667**
12	0.635**	0.664**
13	0.462**	0.441**
14	0.412**	0.428**
15	0.564**		0.737**
16	0.504**		0.730**
17	0.545**		0.780**
18	0.505**		0.651**

Notes:

a A subscale of SHAI.

** P<0.01.

Abbreviations: SHAI, Short Health Anxiety Inventory; IL, Illness Likelihood; NC, Negative Consequences.

Disclosure

The authors report no conflicts of interest in this work.