The factorial structure of the Hong Psychological Reactance Scale in two Finnish samples

Abstract

Psychological assessment scales need to be psychometrically sound, but previous research on the factorial structure of one of the most common measures of trait reactance, the Hong Psychological Reactance Scale (HPRS), has provided multiple latent structures and contradicting findings, and the statistical methodology of previous research can be criticized. These issues raise questions regarding the interpretability and usefulness of the HPRS as a measure of trait reactance. In this two-part study, we followed contemporary statistical recommendations and tested the fit of 12 previously observed factor solutions of the HPRS using confirmatory factor analysis in two relatively large Finnish samples (n = 624 & 518). Furthermore, we tested what factor structure was supported by a data-driven exploratory factor analysis approach. Both our confirmatory and exploratory factor analyses supported a one-factor solution for the HPRS. However, in the exploratory analyses, the single factor accounted for only approximately 40% of the total variance. Our results also indicated that a reduced scale provided the best fit in our confirmatory analyses. Combined, these findings imply that the HPRS could use some improvement. Finally, to shed more light on the reactance construct and the construct validity of the HPRS, we replicated previous research showing negative associations between trait reactance and the Big Five Agreeableness and Conscientiousness traits.

Keywords:

• reactance
• Hong psychological reactance scale
• Big Five
• trait
• factor analysis
• latent structure

1. Introduction

Brehm (1966) defined psychological reactance as a motivational state that is triggered when a person perceives that their freedom of behavior is threatened. Later research has suggested that this motivational arousal (i.e., reactance) is best understood as an intermingling of affect (anger) and negative cognitions (Dillard & Shen, 2005; Quick et al., 2012; Rains, 2013). As a consequence of experiencing reactance, a person will want to restore or defend their perceived freedom. In this context, freedom refers to behaviors that a person values and feels entitled to, and consequently individuals will differ in what restrictions they consider threatening (Brehm & Brehm, 1981).

During its 50-year existence, psychological reactance theory has garnished substantial scientific inquiry, and it has impacted and received input from both clinical psychology and communication research (Rosenberg & Siegel, 2018). Although reactance was originally conceptualized as a temporary state, it was later reconceptualized as also reflecting a stable trait that predicts how prone a person is to experiencing reactance (Rosenberg & Siegel, 2018). However, the validity of trait reactance has been questioned by critics who argue that reactance is context-bound and therefore not stable across situations (Miron & Brehm, 2006; Shoham et al., 2004; see also Silvia, 2006). On the other hand, the Hong Psychological Reactance Scale (HPRS, Hong & Page, 1989), a measure of trait reactance, has shown an acceptable 6-week test-retest reliability (Hong & Faedda, 1996), and trait reactance measures possess utility, as they have been associated with several important outcomes in, for example, therapeutic treatment (Beutler et al., 2002), self-reported work performance (Yost et al., 2019), and health behavior (e.g., Miller et al., 2006; Miller & Quick, 2010). For example, Hornsey et al. (2018) observed in a large multinational study that trait reactance was positively associated with antivaccination attitudes. The authors suggest that knowledge of such attitudinal barriers could help in creating and providing tailor-made interventions that take this predisposition into account (see also, e.g., Finkelstein et al., 2020).

1.1. The Hong Psychological Reactance Scale and the current two-part study

As trait reactance has been shown to have both scientific and practical utility, and as trait reactance scales are widely used, it is important to ascertain that the assessment scales are psychometrically sound. Hence, we have opted to scrutinize the HPRS that is one of the most commonly used trait reactance scales, as previous research has provided multiple latent structures for the scale. These contradicting findings raise questions concerning the interpretability of the scale and the reactance trait more generally.

The HPRS was developed from an English translation of Merz’s Psychological Reactance Scale (Tucker & Byers, 1987; see also Hong & Ostini, 1989; Merz, 1983). For the following rundown of previous research on the factorial structure of the HPRS, please use Figure 1 as a visual aid and see Supplementary material A for exact scale items. The HPRS originally included 14 Likert-scale items that produced a four-component orthogonal structure: (1) freedom of choice, (2) conformity reactance, (3) behavioral freedom, and (4) reactance to advice and recommendations (Hong & Page, 1989). Later work by Hong and Faedda (1996) led to a refined 11-item version. However, the orthogonal four-factor structure of the HPRS has been challenged in several more recent studies, which have collectively produced a multitude of candidate models for the underlying structure of the HPRS. Using confirmatory factor analysis (CFA), Thomas, Donnell, and Buboltz (2001) observed that an oblique four-factory structure provided a better fit than the originally proposed orthogonal structure. Shen and Dillard (2005) argued for the addition of a unidimensional second-order factor. Jonason and Knowles (2006) argued for a one-factor solution (see also Jonason et al., 2010). Brown et al. (2011; see also Yost & Finney, 2018) and Moreira et al. (2019) argued for somewhat different bifactor models, while De las Cuevas et al. (2014) observed that a two-factor model (cognitive, affective) provided the best fit in a sample of psychiatric outpatients. Regarding statistical methodology, Hong’s original work has been criticized for implementing principal components analysis with orthogonal rotation¹ (Brown et al., 2011; Jonason & Knowles, 2006). Furthermore, only one of the relevant studies (Moreira et al., 2019) report using estimation methods appropriate for ordinal data (such as robust WLS; see recommendations by e.g., Brown, 2015) and parallel analysis in deciding how many factors to extract in exploratory analyses (for recommendations, see Baglin, 2014; Hayton et al., 2004; Ledesma & Valero-Mora, 2007; Timmerman & Lorenzo-Seva, 2011).

Figure 1 Previously reported factor-structures for the Hong Psychological Reactance Scale. Numbers 1–14 represent the scale items.

A = Hong and Page (1989). Principal component analysis on 14-item scale, n = 257.

B = Hong (1992). Principal component analysis on 14-item scale, n = 462.

C = Hong and Faedda (1996). Principal component analysis on 11-item scale, n = 3085.

D = Thomas et al. (2001 A). Confirmatory factor analysis (CFA) on 14-item scale, n = 905.

E = Thomas et al. (2001 B). CFA on 11-item scale, n = 539.

F = Shen and Dillard (2005). CFA on 11-item scale, n = 621.

G = Jonason and Knowles (2006 A). CFA on Tucker and Byers (1987) 18-item scale (only 14 items included here), n = 288.

H = Jonason and Knowles (2006 B). CFA on Tucker & Byers’ (1987) 18-item scale (reduced scale, only 14 items included here), n = 288.

I = Jonason et al. (2010). Principal component analysis and CFA on 18-item scale (Tucker & Byers, 1987) (reduced scale, only 14 items included here), n = 240.

J = Brown et al. (2011). CFA on 14-item scale, n = 1286.

K = De las Cuevas et al. (2014). Principal component analysis and CFA on 14-item scale, n = 354.

L = Moreira et al. (2019). CFA on 14-item scale, n = 1301 & 327 (adolescents).

The present two-part study implemented the abovementioned methodological recommendations and investigated the factorial structure of the HPRS in two separate Finnish samples. In Study 1, the main aim was to investigate the factorial structure of the HPRS in a sample of Finnish parents with both confirmatory and exploratory factor analyses. Study 2 focused on investigating the HPRS from a methodological and generalizability-related perspective by testing how minor alterations in wording in some of the HPRS items affected response distributions and the factor structure of the scale in a separate sample of Finnish participants recruited via Facebook. Additionally, in Study 1, a secondary aim was to investigate the construct validity of the HPRS by replicating previous research that had shown negative associations between the HPRS and especially the Agreeableness and Conscientiousness Big Five personality traits. These negative associations were hypothesized to result from more agreeable individuals being more accepting towards relinquishing personal freedom due to their cooperative and compliant nature. On the other hand, those lower in Conscientiousness tend to be less responsible and dependable, which could also be related to a higher dislike of rules and regulations that should be relevant for reactance-propensity (Yost & Finney, 2018). This replication was motivated by the need to establish a nomological network of trait reactance, that is, demonstrate how trait reactance is related to other established individual difference variables (Dowd & Wallbrown, 1993; Seemann et al., 2005).

2. Study 1

In Study 1, we assessed the fit of previously suggested factor structures of the HPRS in Finnish parents, which is a hitherto unstudied population within this context. Additionally, we attempted to replicate previous research indicating that trait reactance is negatively associated with the Big Five traits Agreeableness and Conscientiousness (Yost & Finney, 2018).

2.1. Method

2.1.1. Ethics statement

The study was approved by the Ethics Committee of the Hospital District of Southwest Finland. Informed consent was obtained from all participants.

2.1.2. Participants and procedure

The HPRS and Ten Item Personality Inventory (TIPI; Gosling et al., 2003) were included in a larger survey study that investigated vaccine attitudes among Finnish parents. Therefore, all participants were parents or caregivers to at least one child that was younger than 4.5 years. The survey invitation was sent in the end of May 2018 per mail to 3401 participants in the FinnBrain Birth Cohort Study (www.finnbrain.fi) (Karlsson et al., 2018). Included in the invitation was a participant-specific web-address with which the participant accessed the online survey.² The entire survey took approximately 20 minutes to complete. Our sample consisted of 739 respondents who had complete HPRS data (disregarding item 11, see section 2.2): 624 respondents completed the survey in Finnish and 115 in Swedish³ (see Table 1 for descriptive information).

Table 1 Descriptive statistics of the subsamples in Study 1.

	Finnish speaking respondents	Swedish speaking respondents
	(n = 624)	(n = 115)
Age	M = 36.39, SD = 5.01, Range 25–61	M = 36.66, SD = 4.04, Range 22–45
Gender	67.3% female, 32.7% male	49.6% female, 50.4% male
Education^a	22.4% Mid & low, 29.3% High/voc, 35.3% High, 13.0% Missing	27.0% Mid & low, 20.0% High/voc, 48.7% High, 4.3% Missing

Note: Even though a respondent completed the survey in a given language, it did not necessarily entail that the person was inept in the other language.

a Mid & low = primary school (or part of), vocational training/apprenticeship, vocational school, high school; High/voc = vocational university; High = undergraduate degree (university), graduate degree (Master’s), licentiate/doctorate.

2.1.3. Measurement scales

We included Finnish and Swedish translations of the 14-item HPRS (Hong & Faedda, 1996; Hong & Page, 1989). The English HPRS was translated into Swedish by the authors of this study, and this Swedish translation was in turn translated into Finnish by a private translation agency. Responses were given on a 5-point Likert-scale (1 = completely disagree, 2 = partly disagree, 3 = neither agree nor disagree, 4 = partly agree, and 5 = completely agree). A higher score indicated higher trait reactance. The Big Five personality traits were assessed with Finnish and Swedish translations of the TIPI (Gosling et al., 2003). The Swedish TIPI was retrieved from the GozLab webpage,⁴ and this Swedish version was translated into Finnish by the same private translation agency that had translated the HPRS. The TIPI consists of ten items (two for each Big Five trait) that are scored on a 7-point Likert-scale (1 = disagree strongly, 2 = disagree moderately, 3 = disagree a little, 4 = neither agree nor disagree, 5 = agree a little, 6 = agree moderately, 7 = agree strongly).

2.1.4. Statistical analyses

As only 115 respondents completed the HPRS in Swedish, we restricted our statistical analyses to the Finnish-speaking subsample. However, we report response distributions of the Swedish-speaking sample in Supplementary material A and discuss their implications, as it sheds light on important methodological considerations. In order to assess the factorial structure of the HPRS, we used both confirmatory and exploratory factor analysis. In all analyses, we only included the 624 respondents who had complete HPRS data (i.e., we allowed no missing values on any of the relevant items).

2.1.4.1. Confirmatory factor analyses

In our CFAs, we tested to what degree the 12 previously observed factor solutions of the HPRS fitted our data (see Figure 1). All CFA models were created to correspond as closely as possible to the models that had been proposed by the respective authors. The items that were specified to load on each factor are displayed in Figure 1. In Models A-C, we tested the three different versions of the four-factor model proposed by Hong and Page (1989), Hong (1992), and Hong and Faedda (1996). These factor solutions were observed using principal component analysis with orthogonal rotation, and therefore, the factor co-variances in these models were fixed to zero. Models D-E tested the two four-factor solutions proposed by Thomas et al. (2001) where the factors were allowed to co-vary. Hence, in these models, we specified the co-variances as freely estimated. In Model F, we tested the second-order factor solution proposed by Shen and Dillard (2005). Models G-I tested the three different one-factor solutions suggested by Jonason and Knowles (2006) and Jonason et al. (2010), who used an 18-item scale (Tucker & Byers, 1987). As the present study employed the 14-item scale, fewer items were specified as indicators. Model J tested the bi-factor solution of Brown et al. (2011, see also Yost & Finney, 2018). In Model K, we tested the two-factor solution proposed by De las Cuevas et al. (2014). Here, factor co-variances were specified as freely estimated. Lastly, in Model L, we tested the bi-factor solution of Moreira et al. (2019). In all models, cross-loadings and error correlations between indicators were fixed to zero. All models were statistically identified with df > 0.

The CFAs were computed with version 0.6-3 of the lavaan package (Rosseel, 2012) in R version 3.5.2 (R Core Team, 2018), see Supplementary material B for R script.⁵ As the fitting function, we used robust WLS (WLSMV), which is an estimation method suitable for ordinal and non-normal data. When indicators are measured on an ordinal scale, estimators developed for continuous data, such as maximum likelihood (ML), can result in underestimation of relationships between indicators as well as distorted test statistics. There is also a higher risk of obtaining artefactual factors due to irrelevant item qualities, such as level of difficulty or extremeness (Brown, 2015, p. 353). Unlike ML that uses the observed covariance matrix, WLSMV analyzes the asymptotic covariance matrix based on the polychoric correlations among the indicators. The polychoric correlations assume that the construct measured on the ordinal scale has an underlying continuous and normal distribution. The correlations utilized in WLSMV estimation are, thus, the correlations between these underlying variables, and not between the observed variables.

As indicators of model fit, we used the comparative fit index (CFI), the Tucker-Lewis index (TLI), the root mean square error of approximation (RMSEA), and the standardized root mean square residual (SRMR). For both CFI and TLI, we considered a cutoff of .95 as indicative of good model fit, for RMSEA, .06, and for SRMR, .08 (Hu & Bentler, 1999).

2.1.4.2. Exploratory factor analysis

Due to the multitude of candidate models, limited replications, and possible methodological shortcomings in relevant previous articles, we also wanted to explore what factor structure we would obtain by using a data-driven exploratory factor analysis approach. For the exploratory factor analysis, we used the FACTOR program (Lorenzo-Seva & Ferrando, 2006, 2013) and followed recommendations to use the polychoric correlation matrix and minimum rank factor analysis, as well as parallel analysis in defining the number of factors to extract (Baglin, 2014).

2.2. Results

We noticed that item 11 of our translated HPRS consistently showed low factor loadings (standardized loadings between .128 and .258 depending on the model). Upon closer inspection, it became apparent that the translated item differed from the original English version. The English item reads, “I resist the attempts of others to influence me” (Hong & Page, 1989),⁶ while our Finnish translation was more akin to “I am able to resist the attempts of others to influence me” [emphasis added]. Due to the low factor loadings and difference in wording, item 11 was not included in any analyses in Study 1.

2.2.1. Confirmatory factor analyses

Table 2 depicts the model fit indices for the 12 previously observed HPRS factor solutions (Models A-L). Presently, six of the models could not be empirically identified, and none resulted in good model fit. The empirically underidentified Models C-F all included factors with only two indicators (in some instances because we excluded item 11), which is a possible source for the improper solutions produced by these models, such as out-of-range factor correlations and indicator variances. The bifactor models (Model J⁷ & L) both produced an out-of-range value in the form of a negative factor variance, indicating misspecification of the model. The four-factor solutions in Model A and B, where factors were specified as uncorrelated, provided the worst fit to the data. The best fitting models were the reduced one-factor Models H and I (Model H is depicted in Figure 2), although they reached good model fit on only one of the fit indices (SRMR < .08). As the identified models were not nested, we did not conduct statistical comparison in fit between models.

Figure 2 Factor loadings in the best-fitting confirmatory model H of the Hong Psychological Reactance Scale.

Table 2 Fit statistics of previous Hong Psychological Reactance Scale models.

Model	df	χ^2	CFI	TLI	RMSEA	SRMR
A – Hong and Page (1989)	65	3435.65	.21	.06	.29	.30
B – Hong (1992)	65	3471.33	.21	.05	.29	.30
C – Hong and Faedda (1996)	Model not empirically identified
D – Thomas et al. (2001)	Model not empirically identified
E – Thomas et al. (2001)	Model not empirically identified
F – Shen and Dillard (2005)	Model not empirically identified
G – Jonason and Knowles (2006)	65	564.07	.88	.86	.11	.08
H – Jonason and Knowles (2006)	27	202.23	.92	.90	.10	.07
I – Jonason et al. (2010)	27	241.60	.91	.88	.11	.07
J – Brown et al. (2011)	Model not empirically identified
K – De las Cuevas et al. (2014)	64	535.57	.89	.87	.11	.08
L – Moreira et al. (2019)	Model not empirically identified

Note: See Figure 1 for detailed depictions of the models. df = degrees of freedom, χ² = chi square, CFI = Comparative Fit Index, TLI = Tucker-Lewis Index, RMSEA = Root Mean Square Error of Approximation, SRMR = Standardized Root Mean Square Residual. N = 624.

2.2.2. Exploratory factor analysis

The Kaiser-Meyer-Olkin test of sampling adequacy was 0.89 and Bartlett’s test of sphericity was significant, χ²(78, N = 624) = 2085.9, p < .001, which suggested that the data were adequate for factor analysis. Parallel analysis indicated that only one factor be extracted, and that single factor explained 62.1% of the common variance and 38.0% of the total variance (see Table 3 for factor loadings and communalities).

Table 3 Factor loadings and communalities in the one-factor Hong Psychological Reactance Scale.

	Factor loading	Communality
Item 1	.63	.61
Item 2	.52	.54
Item 3	.74	.82
Item 4	.37	.36
Item 5	.59	.63
Item 6	.73	.73
Item 7	.55	.57
Item 8	.62	.48
Item 9	.74	.78
Item 10	.58	.53
Item 12	.57	.60
Item 13	.63	.61
Item 14	.66	.71

Note: N = 624.

As several of the previous exploratory analyses of the factorial structure of the HPRS have used the Pearson correlation matrix, which tends to underestimate the associations between ordinal-scale items (Gadermann et al., 2012; Holgado-Tello et al., 2010), we present a comparison of the polychoric and Pearson correlation matrices for our data in Table 4. As can be observed, the polychoric correlations tend to be higher than the Pearson correlations.

Table 4 The polychoric (below diagonal) and Pearson (above diagonal) correlation matrices of the Hong Psychological Reactance Scale.

	Item 1	Item 2	Item 3	Item 4	Item 5	Item 6	Item 7	Item 8	Item 9	Item 10	Item 12	Item 13	Item 14
Item 1	–	.345	.462	.175	.272	.314	.233	.256	.414	.223	.184	.347	.391
Item 2	.407	–	.402	.138	.174	.253	.247	.243	.286	.163	.225	.301	.219
Item 3	.600	.524	–	.084	.224	.276	.210	.283	.559	.252	.213	.357	.378
Item 4	.187	.162	.128	–	.312	.374	.252	.205	.071	.168	.226	.130	.108
Item 5	.307	.198	.316	.348	–	.477	.389	.311	.253	.307	.289	.271	.167
Item 6	.369	.296	.377	.441	.555	–	.394	.421	.337	.409	.355	.336	.271
Item 7	.264	.292	.301	.306	.475	.481	–	.334	.236	.245	.265	.163	.177
Item 8	.298	.270	.376	.241	.367	.492	.394	–	.332	.360	.374	.291	.289
Item 9	.516	.361	.685	.106	.334	.438	.316	.426	–	.302	.238	.401	.420
Item 10	.266	.205	.333	.214	.377	.491	.316	.425	.389	–	.302	.269	.333
Item 12	.212	.251	.277	.267	.329	.418	.311	.413	.307	.351	–	.373	.325
Item 13	.400	.344	.455	.163	.314	.403	.201	.349	.513	.325	.434	–	.344
Item 14	.483	.291	.506	.141	.228	.352	.230	.370	.546	.414	.413	.439	–

Note: The polychoric correlation matrix was calculated with the FACTOR program, and the Pearson correlation matrix with SPSS version 25. Item 11 of the Hong Psychological Reactance Scale was not included for this sample. N = 624.

2.2.3. The HPRS and the Big Five

We assessed the association between HPRS trait reactance and the Big Five traits Agreeableness and Conscientiousness by specifying a three-factor model including the factors Reactance, Agreeableness, and Conscientiousness where the factor co-variances were freely estimated. For the reactance factor, we chose to implement Model H as it showed the best fit (although it was only marginally better than Model I, see Table 2). Based on modification indices and theoretical considerations,⁸ we included an error correlation between items 4 and 6. The three-factor model fit the data moderately well, df = 61, χ² = 233.73, CFI = .94, TLI = .92, RMSEA = .07, SRMR = .06 (see Figure 3).

Figure 3 Model depicting the associations between latent variables for trait reactance (Hong Psychological Reactance Scale, reduced ancillary model), Agreeableness, and Conscientiousness (Ten Item Personality Inventory). N = 624. Missing data – Sympathetic, Warm: 0.80%; Dependable, Self-disciplined: 0.32%; Disorganized, Careless: 0.64%; Number of missing patterns: 6.

2.3. Discussion

In the present, relatively large, sample (N = 624), both confirmatory and exploratory factor analyses of the HPRS supported a one-factor solution. This finding contradicts some previous results suggesting a multifactorial structure for the scale (De las Cuevas et al., 2014; Hong & Faedda, 1996; Thomas et al., 2001), but supports (Jonason & Knowles, 2006) or is partly in line with others who advocate the use of a unitary HPRS score (Brown et al., 2011; Moreira et al., 2019; Yost & Finney, 2018). This discrepancy may stem from our use of the polychoric correlation matrix, while most previous studies have assumedly used the Pearson correlation matrix and thus, potentially, underestimated the associations between the ordinal-scale HPRS items.⁹ Furthermore, by relying on the Kaiser criterion and/or a Scree test, previous exploratory research might have overestimated the number of factors to retain (we would have retained two factors according to the Kaiser criterion). In our exploratory analysis, we implemented parallel analysis that is considered a better method of deciding how many factors to retain (Ledesma & Valero-Mora, 2007).

It is, however, important to observe that our best-fitting confirmatory model was a reduced model that was based on a model by Jonason and Knowles (2006).¹⁰ This model did not include items 3, 5, 7, and 10, it achieved good model fit on only one of the fit indices, and it departs from the typical 14- or 11-item versions. The absence of some of the items could in part explain why the reduced unidimensional model showed the best fit, as previous research has suggested that trait reactance is a unidimensional construct, but that the HPRS artificially produces multiple factors due to similar item wording and/or content (Moreira et al., 2019; Yost & Finney, 2018). Therefore, by removing some of the items, the wording/content-specific overlap between the remaining items might have been reduced. Be that as it may, combined, these results suggest that the HPRS is not an “optimal” scale in its current state.

Our results replicate the findings of Yost and Finney (2018) in that trait reactance was significantly negatively associated with both the Agreeableness and Conscientiousness traits. However, in the present study, the associations were stronger than in the study by Yost and Finney (2018), which could be a result of the TIPI that only has two items for each Big Five trait (Yost & Finney, 2018, used the Big Five Inventory that comprises nine items for both Agreeableness and Conscientiousness, John & Srivastava, 1999). This could potentially have exaggerated the presently observed associations, especially in the case of the Agreeableness-reactance association, as the Agreeableness item “Critical, Quarrelsome” is akin to some of the HPRS items (especially item 2, see Supplementary material A). Nevertheless, considering the results of both the present study and that of Yost and Finney (2018), it is clear that these traits (especially reactance and Agreeableness) have quite a lot in common.

A limitation to the generalizability of our current results is the fact that we used translations instead of the original HPRS.¹¹ This could have affected the present results as we found evidence that some of the translated items differed from the original in semantic nuance. This was most apparent in item 14, where the Finnish version had a word for “submit” that mostly corresponds to “agree” or “comply” in English. The Swedish translation, on the other hand, corresponded more strongly to the original English version, and interestingly, the two language groups’ (Finnish & Swedish) response distributions on this item were very different (see Supplementary material A). This suggests that the wording might have affected the response distributions. Therefore, in Study 2, we attempted to rectify the discrepancies in wording between our Finnish translation and the original English version, and tested whether it affected response distributions and the currently observed unidimensional factor structure of the HPRS.

3. Study 2

In Study 2, we edited some of the Finnish HPRS items so that they would more exactly reflect the original English version (see Supplementary material A), and tested if this affected the response distributions and factor structure of the scale.

3.1. Method

3.1.1. Ethics statement

The study was approved by the Joint Ethics Committee of the Departments of Psychology and Logopedics, Åbo Akademi University. Informed consent was obtained from all participants.

3.1.2. Participants and procedure

The 14-item HPRS was included in a larger study that investigated sexuality and perceived mate value, among other topics. Participants were recruited with the help of a Facebook ad that was directed at Finnish speaking Facebook users. Participants completed the survey online in SurveyAnalytics. After the first recruitment-wave, the survey had been completed by a higher number of women than men, and therefore, only men were enrolled in a subsequent recruitment-wave. After excluding those participants who did not provide answers to the 14-item HPRS, the final sample consisted of 518 respondents (see Table 5 for descriptive information).

Table 5 Descriptive information of the sample recruited via Facebook.

Age	M = 33.8, SD = 9.3, Range 18–77
Gender	50.4% female, 48.1% male, 1.5% other
Education^a	62.6% Mid & low, 21.2% High/voc, 16.2% High

Note: ^aMid & low = primary school, vocational training/apprenticeship, vocational school, high school; High/voc = vocational university; High = undergraduate degree (university), graduate degree (Master’s), doctorate. Participant age calculated from year of birth. N = 518.

3.1.3. The HPRS

To better match the English original version of the HPRS, we made minor changes in the wording in altogether ten of the fourteen Finnish items. The response distributions, exact items, and descriptions of the changes we made, are presented in Supplementary material A.

3.1.4. Statistical analyses

In order to assess the factorial structure of the HPRS, we made identical confirmatory and exploratory factor analyses as in Study 1. Note that item 11 was included in the analyses in Study 2.

3.2. Results

A visual inspection of the response distributions showed that the Facebook sample in Study 2 had higher HPRS trait reactance scores than the sample in Study 1 (see Supplementary material A). This pattern of results is also true for the four items that were identical in both studies.

3.2.1. Confirmatory factor analyses

Overall, the results are very similar to Study 1 (compare Tables 2 and 6). Again, Models C-F and J were empirically underidentified. However, in Study 2, the bifactor Model L converged, but it did not provide as good measures of model fit as the reduced one-factor model that again provided the best fit to the data (Figure 4).

Figure 4 Factor loadings in the best-fitting confirmatory model H of the Hong Psychological Reactance Scale.

Table 6 Fit statistics of eleven previous Hong Psychological Reactance Scale models.

Model	df	χ^2	CFI	TLI	RMSEA	SRMR
A – Hong and Page (1989)	77	3926.31	.19	.04	.31	.32
B – Hong (1992)	77	3748.92	.23	.08	.30	.31
C – Hong and Faedda (1996)	Model not empirically identified
D – Thomas et al. (2001)	Model not empirically identified
E – Thomas et al. (2001)	Model not empirically identified
F – Shen and Dillard (2005)	Model not empirically identified
G – Jonason and Knowles (2006)	77	497.74	.91	.90	.10	.07
H – Jonason and Knowles (2006)	35	147.22	.96	.94	.08	.05
I – Jonason et al.(2010)	27	185.71	.93	.91	.11	.06
J – Brown et al. (2011)	Model not empirically identified
K – De las Cuevas et al. (2014)	76	469.96	.92	.90	.10	.06
L – Moreira et al. (2019)	68	378.862	.93	.91	.09	.06

Note: See Figure 1 for detailed depictions of the models.

3.2.2. The exploratory factor analysis

The Kaiser-Meyer-Olkin test of sampling adequacy was 0.91 and Bartlett’s test of sphericity was significant, χ²(91, N = 518) = 2203.9, p < .001, which suggested that the data were adequate for factor analysis. Parallel analysis indicated that only one factor be extracted, and that single factor explained 66.1% of the common variance and 40.2% of the total variance (see Table 7 for factor loadings and communalities). As in Study 1, we have included the polychoric and Pearson correlation matrices (see Table 8).

Table 7 Exploratory factor analysis loadings and communalities of the one-factor Hong Psychological Reactance Scale.

	Factor loading	Communality
Item 1	.58	.51
Item 2	.48	.50
Item 3	.74	.92
Item 4	.43	.42
Item 5	.68	.68
Item 6	.68	.72
Item 7	.59	.52
Item 8	.64	.62
Item 9	.75	.73
Item 10	.67	.62
Item 11	.70	.66
Item 12	.60	.58
Item 13	.66	.50
Item 14	.59	.53

Note: N = 518.

Table 8 The polychoric (below diagonal) and Pearson (above diagonal) correlation matrices of the Hong Psychological Reactance Scale.

	Item 1	Item 2	Item 3	Item 4	Item 5	Item 6	Item 7	Item 8	Item 9	Item 10	Item 11	Item 12	Item 13	Item 14
Item 1	–	.292	.519	.139	.263	.288	.162	.208	.412	.304	.284	.313	.311	.384
Item 2	.352	–	.442	.139	.259	.149	.186	.211	.330	.247	.239	.244	.263	.231
Item 3	.585	.516	–	.176	.307	.315	.256	.291	.609	.369	.364	.290	.439	.423
Item 4	.170	.166	.211	–	.253	.351	.253	.242	.157	.268	.246	.206	.208	.223
Item 5	.322	.297	.370	.312	–	.448	.445	.398	.426	.332	.407	.337	.347	.255
Item 6	.337	.173	.374	.398	.524	–	.435	.439	.328	.414	.414	.348	.330	.270
Item 7	.233	.224	.311	.326	.533	.506	–	.412	.292	.247	.315	.280	.296	.194
Item 8	.265	.261	.350	.284	.468	.513	.496	–	.356	.360	.353	.323	.355	.289
Item 9	.478	.383	.674	.200	.497	.394	.363	.424	–	.432	.432	.323	.467	.414
Item 10	.360	.288	.429	.305	.395	.477	.306	.420	.500	–	.491	.321	.337	.351
Item 11	.322	.274	.409	.295	.474	.492	.384	.439	.484	.565	–	.437	.399	.346
Item 12	.352	.275	.339	.234	.385	.394	.339	.369	.377	.365	.493	–	.375	.347
Item 13	.366	.314	.508	.261	.415	.405	.378	.422	.538	.410	.450	.433	–	.339
Item 14	.428	.275	.477	.250	.296	.315	.257	.336	.470	.402	.385	.394	.402	–

Note: The polychoric correlation matrix was calculated with the FACTOR program, and the Pearson correlation matrix with SPSS version 25. N = 518.

3.3. Discussion

The results of our confirmatory and exploratory factor analyses in Study 2 replicated the findings we observed in Study 1, indicating that the underlying latent structure of the HPRS is best described as unidimensional.

The fact that the participants’ trait reactance scores were higher in Study 2 than in Study 1 also on items that were identical in both studies suggests that this difference is not solely due to the changes in wording that was made in some of the HPRS items (see Supplementary material A). Therefore, it is difficult to conclude whether the revised wording had affected the response distributions in Study 2. However, in item 14 of the HPRS, it seems obvious that the use of the word “comply” (in Study 1) or “submit” (in Study 2) in our Finnish translations has had an impact on the response distributions so that the item is negatively skewed in Study 1, but approximately evenly distributed in Study 2 (see Supplementary material A). This conclusion is backed up by the fact that the Swedish subsample in Study 1 and the Finnish sample in Study 2 have similar response distributions and more closely corresponding wording on that specific item.

4. General discussion

Psychological assessment scales need to be psychometrically sound, however, even though the HPRS is one of the most commonly used measures of trait reactance, previous research has failed to converge on a common factorial structure for the scale (see e.g., De las Cuevas et al., 2014). A possible reason for this variability may be that many previous studies on the topic have potentially been hampered by suboptimal statistical methods (Brown et al., 2011). This issue raises questions regarding the scale’s reliability and also about practical issues related to how the HPRS should be used and interpreted. In the present two-part study involving two relatively large Finnish samples (n = 624 & 518), we wanted to ameliorate the situation by testing the fit of 12 previously suggested latent structures of the HPRS when using appropriate statistical methods. Furthermore, to shed light on the reactance construct and the construct validity of the HPRS, we attempted to replicate previous research that showed negative associations between the HPRS and the Agreeableness and Conscientiousness Big Five traits (Yost & Finney, 2018).

Our confirmatory and exploratory factor analyses in both Studies 1 and 2 indicated a one-factor solution for the HPRS. The present one-factor solution supports the results of Jonason and Knowles (2006; see also Jonason et al., 2010) and partly Brown et al. (2011; see also Yost & Finney, 2018) and Moreira et al. (2019), but contradicts several other previous studies that have obtained multifactorial structures for the scale (including Hong’s original work). As discussed in the Discussion of Study 1, the previous discrepant findings could be the result of differing statistical methods that might have resulted in a higher number of factors in some of the previous studies. The present results suggest that the HPRS is best interpreted as unidimensional. Simultaneously, however, it seems that the scale could be improved, as the confirmatory analyses supported a reduced scale that did not match the typical 14- or 11-item versions. Also, in the exploratory analyses, the single factor accounted for “only” 40% of the total variance. Moreover, previous research has shown less-than-ideal convergent validity for the Questionnaire for Measuring Psychological Reactance, which is an English translation of Merz’s reactance scale (1983) and relative/precursor of the HPRS. The correlations between that scale and the Therapeutic Reactance Scale (another measure of trait reactance, Dowd et al., 1991) have ranged from .50 to .58 (Buboltz et al., 1999; Dowd & Wallbrown, 1993; Dowd et al., 1994), which is less than a recommended .70 (Carlson & Herdman, 2012). On the other hand, Moreira et al. (2019) reported a high association (β = .89) between the HPRS and the Therapeutic Reactance Scale in a structural equation modeling analysis with a large sample of adolescents, but the model did not show adequate model fit (CFI = .96, TLI = .87, RMSEA = .16, & SRMR = .08). The same authors reported a markedly smaller association (r = .50) for composite scores in a different smaller sample of adolescents (Moreira, et al., 2019). This potentially indicates that the HPRS is not a comprehensive measure of trait reactance, and further research is needed to clarify this issue. A potentially productive method for refining a future trait reactance scale is to expand the research on how specific trait reactance items are related to objectively observable behaviors and outcomes (not only self-reported attitudes, feelings, behaviors, and personality traits), and use those observations, in combination with confirmatory studies, to select items that show temporal stability and predictive value (beyond other traits such as Agreeableness) for outcomes that are relevant from a trait reactance perspective¹² (see also Yost & Finney’s, 2018, recommendations for revising the HPRS items).

Our results corroborate the findings of Yost and Finney (2018), in that trait reactance was negatively associated with both the Agreeableness and Conscientiousness traits (however, see Seemann et al., 2005¹³). Yost and Finney (2018) suggested that because more agreeable individuals tend to be more compliant, they might experience less psychological reactance when their freedoms are threatened or restricted. Individuals lower in Conscientiousness, on the other hand, tend to be less responsible and dependable, which may be related to higher trait reactance due to a greater tendency to dislike rules and regulations. These results fit with previous personality profile accounts of predispositionally reactant individuals that describe them as aggressive, dominant, persuasive, defensive, and quick to take offense (Buboltz et al., 1999; Dowd & Wallbrown, 1993); and “less likely to follow social norms and rules and [who] may be somewhat careless about fulfilling duties and obligations” (Dowd et al., 1994, p. 609). Moreover, previous research has indicated negative associations between trait reactance, as assessed by the Therapeutic reactance scale, and the persistence, cooperativeness, and, to some extent, self-directedness dimensions¹⁴ (Inman et al., 2019) of the psychobiological model of personality (Cloninger et al., 1993). This is to be expected, as Cloninger’s Temperament and Character Inventory (a self-rating scale that assesses the dimensions of the psychobiological model, Cloninger et al., 1993) has shown significant overlap with the Big Five (Capanna et al., 2012; De Fruyt et al., 2000; Garcia, 2012; Ramanaiah et al., 2002). More specifically, persistence and self-directedness have been significantly associated with Conscientiousness (r’s .46–.57 and .40–.45 respectively), and cooperativeness with Agreeableness (r’s .51–.64). Furthermore, trait reactance has been positively associated, albeit somewhat weakly and tentatively, with procrastination (Malatincová, 2015) that in turn has been negatively associated with Conscientiousness (r = −.63; van Eerde, 2003). Combined, these results provide some convergent validity for the currently replicated trait reactance-Big Five associations.

Interestingly, the sample recruited via Facebook in Study 2 reported substantially higher levels of trait reactance than the sample in Study 1. Based on the present study, it is difficult to draw conclusions on the reasons behind this difference. However, a feasible explanation is participant characteristics. It is possible that the parents that are active in the FinnBrain Birth Cohort Study, which includes measurements during multiple time points over several years, are more compliant by nature than the participants in Study 2 who were recruited via Facebook (who only completed one survey). Predispositionally reactant individuals may be less inclined to participate in a multi-year longitudinal study that could be perceived as restricting personal freedom. Another possible reason is that parents to small children (Study 1) could consider reactance as socially undesirable, due to the aggressive nature of reactance (e.g., Buboltz et al., 1999), and therefore underreport their tendencies to experience reactance. Unfortunately, we did not collect data on whether the participants in Study 2 had children, and therefore we could not test if parenthood was associated with lower HPRS scores. On the other hand, an alternative explanation is that the sample recruited via Facebook was for some reason higher in trait reactance than people on average, and that the parents in Study 1 more closely represent the general population in this regard.

The main limitation/concern in the present studies are the potential issues with comparability and generalizability due to our Finnish translation of the English HPRS. Our studies indicated that a rather subtle difference in wording in especially item 14 of the HPRS strongly affected the response distributions on that item. Therefore, the current results could be scale/translation-specific. However, even though we made changes in some of the HPRS items, we still obtained strongest support for the same unidimensional factor-structure in both Studies 1 and 2, which suggests that this underlying latent structure appears rather robust.

In conclusion, our results support a unidimensional latent structure for the HPRS, and we also observed that this latent factor was significantly negatively associated with the Agreeableness and Conscientiousness traits. However, as pointed out also by others, the HPRS does not seem optimal in its current state (e.g., Jonason & Knowles, 2006; Yost & Finney, 2018).

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1 Both principal components analysis and (exploratory) factor analysis are data reduction methods. However, while principal components analysis is a linear combination of variables that maximizes the retained variance, exploratory factor analysis attempts to identify latent variables that explain the relationships among the observed variables (Preacher & MacCallum, 2003). Orthogonal rotation forces the components or factors to be uncorrelated, which is rarely warranted in this kind of psychological research.

2 Due to a technical error, some responses were not registered for all participants, and in October 2018, a second batch of 3101 invitations were sent (including 448 letters with requests to redo the survey).

3 Finland is officially a bilingual nation that has a Swedish-speaking minority that constitutes approximately 5% of the total population.

4 https://gosling.psy.utexas.edu/scales-weve-developed/ten-item-personality-measure-tipi/

5 Due to data privacy, the raw data for Study 2 cannot be shared.

6 Copyright© by Sage Publishing, reprinted with permission.

7 We also tested the slightly different bifactor model reported in Yost and Finney (2018). In that model, the factor pattern coefficients were constrained to be equal for items 5 and 9 (instead of using a correlated error), but this model was not empirically identified.

8 Both item 4 and 6 are related to dependence, and their similarity was possibly pronounced in our translation as both items included a reference to independence from others (see Supplementary material A).

9 The participants in Moreira et al. (2019) consisted of adolescents, which could explain why they observed best support for a different latent structure.

10 Jonason and Knowles used an 18-item scale (see Tucker & Byers, 1987), while we used the 14-item scale (Hong & Page, 1989).

11 It is also possible that the observed one-factor structure is dependent on the present sample of Finnish parents, but this seems intuitively unlikely. That is, why would the factor structure of the HPRS be more uniform among Finns than among, e.g., Australians, and/or why would it be more uniform among parents to young children than, e.g., college students?

12 Using this method, items in a trait reactance scale would be selected on the basis of their temporal stability and ability to predict (beyond other traits such as Agreeableness) a reactance responses when, e.g., avid smokers are informed that their employers will prohibit smoking during work hours, pacifists are informed that their governments will institute conscription etc.

13 Seemann et al. (200559) found a significant association between trait reactance, as measured by the Therapeutic Reactance Scale (TRS), and Agreeableness, but not between trait reactance and Conscientiousness, which potentially highlights the differences between the TRS and HPRS trait reactance scales.

14 Briefly and somewhat selectively reported, Persistence has been described to reflect perseverance despite frustration and fatigue; Cooperativeness has been described to reflect acceptance of others, agreeability, helpfulness, and compassion; and Self-directedness has been described to reflect self-determination, sense of responsibility, and an ability to control and regulate behavior (Cloninger et al., 1993).