Assessing beliefs about emotion generation and change: The conceptualisation, development, and validation of the Cognitive Mediation Beliefs Questionnaire (CMBQ)

Abstract

The ability to regulate emotions is important for human function and health. That emotion regulation can be achieved through cognitive change is predicated on the notion of cognitive mediation. However, the extent to which individuals believe that their emotions are cognitively mediated (C–M), or in contrast, that their emotions occur via stimulus-response (S-R), is underexplored, and whether C–M and S-R beliefs shape emotion reactivity is not yet known. Research that addresses these empirical needs could inform emotion regulation interventions such as cognitive behavioural therapies (CBTs). The current paper reports the development and initial validity testing of the cognitive mediation beliefs questionnaire (CMBQ). Five studies report the factor structure, the construct and criterion validity, and the test-retest reliability of the CMBQ. The CMBQ was found to have a correlated two-factor structure (C–M change beliefs, and S-R generation beliefs). Higher C–M change beliefs and lower S-R generation beliefs were related to greater emotion regulation, greater thought control ability, higher positive mental health, and lower emotion reactivity. The CMBQ also demonstrated acceptable test-retest reliability. Initial testing indicates that the CMBQ is a valid and reliable questionnaire for psychometric use in adult populations, including those with a diagnosed mental health condition.

Keywords:

• reappraisal
• REBT
• stoicism
• cognitive restructuring
• emotion belief

Clinical or Methodological Significance of this Article: This paper conceptualises, develops, and validity tests a new psychometric for the assessment of emotion beliefs concerning cognitive mediation, a fundamental tenet of CBT. The cognitive mediation beliefs questionnaire (CMBQ) can be used by practitioners and researchers to assess individuals’ endorsement of cognitive mediation to inform the therapeutic approach, and to monitor changes in these beliefs as a result of therapy or intervention work.

Emotion regulation refers to attempts to influence which emotions one has, when one has them, and how one experiences and expresses them (Gross, 2015). Emotion regulation can be recruited for up and down regulation, or maintenance of positive and negative emotions, and can change individual’s emotions across dimensions of valence, arousal, and approach-avoidance (Koole, 2009). Successful emotion regulation supports increased well-being, better social functioning, better coping with stressors, job success (Salovey et al., 2010), psychological health (Kobylińska & Kusev, 2019) and physical health (Sapolsky, 2007).

Amidst the importance of effective emotion regulation, theory has emerged that captures the various ways in which emotion regulation can be achieved. Most prominently, Gross' (2014) process model of emotion regulation incorporates five emotion regulation strategies: situation selection, situation modification, attentional deployment, cognitive change, and response modulation. Cognitive change (or cognitive reappraisal), the modification of one’s appraisal of a situation in order to alter its emotional impact (Gross, 2015), is among the best studied emotion regulation strategies (McRae et al., 2010), and is demonstrably one of the most effective ways to down-regulate negative affect (Boehme et al., 2019). Cognitive change is associated with a gamut of adaptive psychophysiological (e.g., Ray et al., 2010), and neurological (e.g., Ochsner et al., 2004) outcomes. Cognitive change is preferable to expressive suppression (Cutili, 2014) and distraction (e.g., Hayes et al., 2010).

The process model of emotion regulation is relevant to psychopathology (Sheppes et al., 2015), and to psychotherapies implementing cognitive change (e.g., Hofmann et al., 2012). Cognitive change, as presented in Gross' (2014) process model, is predicated on the axiomatic principle imbedded within second-wave cognitive behavioural psychotherapies (CBTs), that cognitions mediate between the environment and our emotional responses (Mahoney, 1974; Ruggiero et al., 2018). In other words, in the face of an adverse situation, our thoughts about the event largely determines our emotional reactions. This proposal is supported by appraisal theories of emotion (e.g., Lazarus, 1999) and emotion regulation (Gross & Thompson, 2007), and by many studies (see Roseman & Smith, 2001, for a review), where with the situational stimuli kept equal, cognitive appraisal is the proximal cause of differential emotion reactivity (e.g., Turner et al., 2014). Thus, in the generation and regulation of emotion, cognitive mediation and cognitive change is paramount, and individuals undergoing CBT learn the A × B = C framework in which thoughts (B) about the situation (A) underpin emotions (C), not the situation or the thoughts alone in isolation. The term “A-C thinking” (i.e., Adversity—Consequence; Turner, 2016) is used to capture the idea that the situation alone causes emotion, and the term “B-C thinking” (Beliefs—Consequences; Turner, 2016), is used to express cognitive mediation, which does not exclude the influence of A (Dryden, 2012), but places B and the centre of emotion reactivity.

As a result, second wave CBTs, especially rational emotive behaviour therapy (REBT; Ellis, 1995), focus largely on direct cognitive change in relation to B, rather than A and C. In other words, rather than changing the situation or directly modulating the emotion, the individual restructures their cognitions, to bring about emotion change. This has the advantage of giving the patient some volition over their emotional reactivity, because they can learn to execute cognitive change autonomously, thus exercising emotional responsibility (Clark, 2013). In sum, cognitive change boasts empirical support within emotion regulation literature (Boehme et al., 2019), and is the backbone of prominent psychotherapies.

Stimulus-Response (S-R) vs. Cognitive Mediation (C–M)

The CBT-derived concepts of A-C and B-C thinking are akin to the concepts of stimulus-response (S-R; e.g., Lazarus, 1999) and cognitive mediation (C–M), respectively. A S-R viewpoint of emotion indicates that emotions are the result of external events, and thus for emotional change to take place, stimulus (or situational) change must take place. A S-R viewpoint could be useful for emotion regulation as it could lead to one actively avoiding certain noxious future situations, and or the removal of oneself from dangerous current situations. It is also an appealing way to think because we can point to harmful external events and ignore our own role in the emotion (Lazarus, 1999). The three components of situation selection, situation modification, and attentional deployment as presented in the process model of emotion regulation (Gross, 2014) are reflective of an S-R viewpoint, because they are predicated on the idea that to regulate emotion, the situation needs to change (via selection or modification) or be ignored. These strategies offer very practical regulation options but are limited due to the impossibility of selecting or modifying the many uncontrollable and unpredictable situations one is likely to encounter, and as such are inherently error-prone (e.g., Gilbert et al., 1998). Situational selection/modification and attentional deployment also encourage avoidance for short-term gains, where emotionally arousing situations are avoided or ignored, ameliorating the emotion in the short-term, only for the emotion to return in the face of a similar situation. Indeed, distraction is considered to be a maladaptive strategy (Trincas et al., 2016), that is useful in the short-term (Ford & Gross, 2018), but can be maladaptive in the long-term (e.g., Sheppes & Gross, 2013).

Relative to a S-R viewpoint, a C–M viewpoint allows greater volition over emotion regulation due to malleability of the content of conscious thought. A C–M viewpoint of emotion indicates that emotion results from cognition (i.e., appraisals, beliefs, thoughts) about events, and thus for emotion change to take place, cognitive change (i.e., reappraisal; Gross, 2015) must take place. Even within uncontrollable situations, evidence indicates that cognitive change is a superior emotion regulation strategy (Doré et al., 2016). Thus, when helping people to develop effective emotion regulation skills via the application of second wave CBTs, a C–M viewpoint is fundamental to the therapeutic mechanisms of change.

Emotion Beliefs

Much research demonstrates support for the effectiveness of cognitive change (e.g., Boehme et al., 2019), but little research has examined the role of explicit beliefs about cognitive mediation in emotion and emotion regulation. This is an important line of enquiry, because individuals differ in how they think about emotions and these varying beliefs are deeply consequential for emotion regulation (Ford & Gross, 2019; Gross & Barrett, 2011). Beliefs about emotion and emotion regulation, or “emotion beliefs” (Ford & Gross, 2019, p. 74), appear to be important for acute outcomes (e.g., emotional experiences), and chronic cumulative outcomes (e.g., well-being; Ford & Gross, 2019). Negative emotion beliefs are a key criterion defining emotional disorders (Bullis et al., 2019), but empirical research lags (Goodman et al., 2020). Two emotion beliefs that are garnering growing interest are, (a) whether emotions are good or bad (“goodness”), and (b) whether emotions are controllable or uncontrollable (“controllability”); neither capture the S-R and C–M viewpoints at the centre of the current paper. Given the potential importance of emotion beliefs for emotion regulation and mental health outcomes (e.g., Tamir et al., 2007), there is a call for researchers to examine the influence of emotion beliefs on psychological health (Ford & Gross, 2018).

In the present paper we propose four superordinate emotion beliefs that capture the S-R and C–M viewpoints, distinct from extant emotion beliefs concepts in the literature. Specifically, we aim to develop a self-report questionnaire that conceptualises and assesses four emotion beliefs:

1. S-R generation beliefs (emotions are caused by events)

2. S-R change beliefs (changes in the situation lead to emotion change)

3. C–M generation beliefs (emotions are cognitively mediated)

4. C–M change beliefs (changes in cognition lead to emotion change).

S-R generation and change beliefs are characterised by the notion that emotion is dependent upon situational events, whilst C–M generation and change beliefs are characterised by the notion that emotion is dependent upon one’s cognitions about situational events. As can be seen above, both S-R and C–M emotion beliefs comprise two related ideas; (a) that emotion is generated by either the situation (S-R generation), or by one’s cognitions about the situation (C–M generation), and that (b) emotion is regulated by changing the situation (S-R change), or by changing one’s cognitions (C–M change). This two-factor approach assumes one factor that involves a set of processes related to the generation of emotion, and a second factor that involves a different set of processes related to the management of emotion (e.g., Campos et al., 2004; Cole et al., 2004).

This two-factor approach holds that, distinguishing emotion from emotion regulation is useful for analysing processes, individual differences, and fashioning clinical interventions (Gross & Thompson, 2007). But, there is a school of thought that emotion regulation cannot be separated because they are tightly intertwined (Gross & Thompson, 2007), interacting at all phases of emotion generation, manifestation, and termination (Campos et al., 2004). Indeed, in psychological construction models, emotions are continually subject to development and change, and in appraisals models, appraisal is considered an iterative process (e.g., Gross & Barrett, 2011). Contemporary social constructionist perspectives indicate that emotions are constituted and constrained by sociocultural factors and contexts. Thus, emotions are determined by the concurrence of cultural influences as well as physiological and cognitive processes (Aranguren, 2017), and emotions are constructions of the world instead of mere reactions to it (Barrett, 2017). For example, calling oneself “angry” would apply if you felt a certain kind of agitation, thought that you had been unjustly injured, and you had a tendency to respond to the situation in some rather strong manner (Harré, 2009). The bodily agitation, situational characteristics, and the responses that define the emotion, are subject to social and cultural variation. In all, emotion generation and regulation may be inseparable (Gross & Barrett, 2011), but treating them as separate for study is favourable (Gross & Thompson, 2007).

Generation and change beliefs are treated as separate emotion beliefs, but there is of course significant overlap. If one holds a S-R generation belief, then one is probably more likely to hold a S-R change belief, since it is the situation that is most salient to the emotion and is thus the focus of change. But, it is of course possible to hold a S-R generation belief, and still believe that cognitive change is needed (C–M change belief), or hold a C–M generation belief, and believe that situational change is needed (S-R change belief). The orthogonality of the proposed emotion beliefs is not presumed and needs to be examined.

Extending the area of emotion beliefs beyond the emergent concepts of goodness and controllability is important for emotion science (Ford et al., 2018; Tamir et al., 2007), because people’s active attempts to manage their emotions might be in part predicated on their beliefs about how emotions occur and how they can be managed (Ford & Gross, 2019). The proposed S-R and C–M beliefs might be important for emotion regulation, because such superordinate emotion beliefs precede emotion regulation attempts (Trincas et al., 2016), and influence the occurrence and effectiveness of emotion regulation attempts (Goodman et al., 2020). For example, a belief that cognitive change cannot yield emotion change, may result in premature stopping of emotion-regulation efforts (Sheppes et al., 2015). Further, there is little justification for reappraising one’s current situation if one does not believe in the underlying veracity of cognitive change (e.g., reappraisal) or if one places responsibility for the emotion on the situation itself.

Our proposal is that beliefs characterised by C–M are more advantageous for emotion regulation, compared to beliefs characterised by S-R. To expand, greater endorsement of C–M beliefs is presumably associated with a greater propensity to effectively regulate emotion, because cognitive change is more likely to be applied. In contrast, greater endorsement of S-R beliefs, where situational change needs to occur in order to regulate the emotions, is presumably associated with poorer emotion regulation, due to the inherent limits in volitional influence over external events, which discounts cognitive change. To the authors’ knowledge, researchers have yet to investigate the utility of the four proposed emotion beliefs.

The Present Study

In order to render the four proposed emotion beliefs operational for research and practice, in this paper we report the development of an item pool that reflects the four beliefs, and we undertake psychometric validation and reliability analyses in order to produce a self-report questionnaire. Extant literature offers self-report questionnaires that assess the perceived ability to regulate emotions (e.g., emotion regulation questionnaire, Gross & John, 2003; thought control ability questionnaire, Williams et al., 2010), and emotion controllability (e.g., implicit theories of emotions scale; Tamir et al., 2007). Previous measures and the concepts they capture are demonstrably valuable for understanding emotion regulation, but do not capture the four emotion beliefs at the centre of the present paper.

Emotion beliefs are an attractive target for clinical intervention because these beliefs are malleable (Molden & Dweck, 2006), and therefore a questionnaire that assesses C–M and S-R beliefs is potentially therapeutically valuable, as it can inform treatment direction, progress, and effectiveness. This is especially the case for second-wave cognitive–behavioural approaches, in which cognitive change is fundamental to the therapeutic process (Hofmann et al., 2007). A client who grasps a C–M viewpoint is more able to exercise some volition over their emotions by modifying their cognitions (Tarricone, 2011); since not grasping a C–M viewpoint essentially undercuts the therapeutic mechanisms of change. Therapeutically, it is important that patients are aware that whilst they cannot always or fully control and sanitise the external environment, they can learn to challenge and change their own cognitions with a view to becoming more psychologically healthy (Turner, 2016). By assessing and monitoring C–M and S-R beliefs it is possible to elucidate the extent to which the client is ready for cognitive change techniques, and to what extent their endorsement of C–M and S-R beliefs contributes to intervention outcomes.

In the present paper, the development and initial validation of a new psychometric questionnaire is reported, that proposes to assess individuals’ S-R and C–M beliefs. We hope to develop a greater empirical understanding of whether, and to what extent C–M and S-R beliefs are relevant to emotional and mental health outcomes. The current paper includes five empirical studies that align with psychometric questionnaire development guidelines for a latent variable approach (Cabrera-Nguyen, 2010). The development and validity testing of the questionnaire follows guidance posited by Boateng et al. (2018). Study 1 reports exploratory factor analysis (EFA), study 2 reports confirmatory factor analysis (CFA), study 3 reports CFA and criterion validity, study 4 reports test-retest reliability, and study 5 reports CFA and criterion validity in a sample of participants diagnosed with a mental health condition. We pose the question: to what extent do individuals’ naïve beliefs about the generation and regulation of their emotions influence their emotion reactivity?

Study 1: Exploratory Factor Analysis (EFA)

Accurate item generation is central to the development of a good psychometric questionnaire (Irwing & Hughes, 2018), so we conducted a six-stage development, review, and refinement process to establish suitable items for inclusion in the CMBQ. See supplementary file 1 for a detailed portrayal of the item generation process employed in the current study. In brief, items reflected C–M (generation and change) and S-R (generation and change) beliefs only, yielding a total of 83 suitable items for analyses. Following psychometric questionnaire development guidelines (e.g., Cabrera-Nguyen, 2010), exploratory factor analysis (EFA) was used to assess the underlying factor structure of the CMBQ (Costello & Osborne, 2005). As recommended (Worthington & Whittaker, 2006), we conducted EFA first, and then followed up with confirmatory factor analyses (CFA) with different samples. For EFA the 83-item CMBQ was created with the stem, “When I experience an unpleasant or unwanted emotion, I believe that … ” Each item in the CMBQ was scored against a five-point Likert-scale from 1 (strongly disagree) to 5 (strongly agree). The details of study 1 were pre-registered on the Open Science Framework (https://osf.io/r872e/?view_only=c4281a1714a842e9964c7686b4da1e72).

Participants

In order to minimise errors and maximise the accuracy and generalizability of population estimates, an a priori participant:item ratio of 10:1 was considered a suitable sample size for EFA (Boateng et al., 2018; Osborne & Costello, 2016). We aimed to recruit a minimum of 830 participants via Prolific, a platform for online research participant recruitment used successfully in past research (e.g., Palan & Schitter, 2018). Only participants who were in full-time employment, were not currently a student, and were based in the U.K., were approached to take part (n > 65,000). Participants were provided with an information sheet before agreeing to take part, and after giving consent, participants could complete the CMBQ using their Smartphones, tablets, laptops, or desktop computers. A total of 919 respondents accessed the online survey platform Qualtrics. A total of 53 respondents’ data were excluded due to poor data quality (straight line responding, unrealistic completion time). The final sample included 866 respondents (Mage = 35.04, SDage = 9.36; female = 432; see supplementary file 5 for demographic information), taking on average 13.58 min (SD = 5.18 min) to complete the CMBQ. Ethical approval was granted from a University ethics committee, and informed consent was gained from participants.

Data Analysis

Data were screened for outliers (standardised z values > 3.29; Hahs-Vaughn, 2016), and outliers were Winsorized (n = 121 from 71,878 cases = .17%; Tokunaga, 2016). Kaiser Meyer-Olkin (KMO = .96) measure and Bartlett’s test of sphericity, X²(3403) = 40,931.67, p < .001, indicated the suitability of the dataset for factor analysis. An EFA using Maximum Likelihood was carried out (Groarke & Hogan, 2018) using SPSS version 25. Detailed EFA procedure can be found in supplementary file 2. In the EFA analysis (see Supplementary file 3 for the initial EFA) 61-items were deleted iteratively for failing to meet item retention criteria, and 22-items across three factors were extracted, accounting for 57.93% of the variance (Table I). EFA revealed two S-R factors (n = 9 and 3-items respectively), and one C–M factor (n = 10-items). The S-R factors were positively correlated (r = .34, p <. 001), and the C–M factor was negatively correlated (p < .001) with the primary S-R factor (S-Ra; r = -.37) and secondary S-R factor (S-Rb; r = -.18). In contrast to study expectations, only items reflecting C–M change and S-R generation beliefs formed acceptable factors. The EFA process revealed that items reflecting C–M generation and S-R change did not form distinct factors, and poor factor loadings meant that all but two items were removed. Only item 83 reflected C–M generation, and only 47 reflected S-R change. Therefore, EFA analyses did fall in line with a two-factor approach to emotion regulation (e.g., Campos et al., 2004), those two factors being C–M change and S-R generation.

Table I. EFA outcomes for the three-factor model, with factor loadings, cross, loadings, and communalities.

Items	Factor-loading	Cross-loading S-Ra	Cross-loading C-M	Cross-loading S-Rb	Communalities	% variance	Loading range	Loading mean	α	Eigen Value	M(SD)
Factor 1: S-Ra						34.13	.64-.75	.70	.90	7.51	2.62(.99)
40	.749	–	−.032	−.052	.560
76	.724	–	−.005	−.064	.504
50	.724	–	−.008	.050	.553
55	.706	–	−.015	.154	.598
35	.696	–	.047	.175	.564
17	.691	–	−.058	−.132	.473
19	.689	–	.023	.118	.525
10	.678	–	.065	.094	.476
47	.638	–	−.076	−.127	.416
Factor 2: C-M						15.54	.60-.75	.70	.91	3.42	3.73(.82)
43	.746	−.071	–	.056	.593
44	.717	.016		.065	.498
45	.715	−.007	–	.044	.508
69	.714	.035	–	.026	.488
20	.713	−.060	–	.018	.542
56	.711	−.028	–	.005	.521
23	.707	−.050	–	.010	.529
80	.703	.079	–	−.052	.466
75	.684	−.030	–	−.057	.500
83	.603	.035	–	−.180	.406
Factor 3: S-Rb						8.26	.66-.72	.70	.76	1.82	3.69(.80)
32	.719	−.021	.019	–	.505
42	.710	.013	−.029	–	.516
51	.661	.144	−.085	–	.447

Note. Total variance explained = 57.93%; Mean α = .86, χ² = 544.763, df = 168, p < .001.

Study 2: Confirmatory Factor Analysis (CFA)

We recruited a separate CFA sample to the EFA sample (e.g., Cabrera-Nguyen, 2010) to test the EFA-informed three-factor structure and associated psychometric properties (e.g., Costello & Osborne, 2005). In addition, we used the CFA to test an alternate two-factor model, for two reasons. First, on inspection of the items comprising the second S-R factor (S-Rb), it was clear that there was little to distinguish them from the items comprising the first S-R factor (S-Ra). Two of the items (42 and 32) in the S-Rb factor include the notion of “a direct link” between life events and emotions, which does not appear in the S-Ra factor. However, item 51 is not distinctive, and is very similar to items 17 and 55 in the S-Ra factor. Second, with the retained items there was an a priori theoretical structure reflecting a two-factor model (S-R generation, and C–M change). Indeed, it is important to apply qualitative judgement to evaluate factor appropriateness in relation to the intended factor structure (e.g., Korlén et al., 2018) and to be guided by theory to provide the results that make the most sense (Osborne, 2014). We wanted to pragmatically arrive at a final set of items that clearly assess theoretically derived emotion beliefs. So, in the CFA we tested the three-factor and two-factor models and compared them to arrive at the most parsimonious solution.

Participants

Researchers suggest a CFA sample size of n = 500 to be very good (Williams et al., 2010), so for the 22-item CMBQ, 490 respondents were recruited. We used the blacklist facility in Prolific to ensure that participants who had participated in the EFA study could not be approached to take part in the CFA study. Of the 490, 43 respondents’ data were excluded due to poor data quality (i.e., straight line responding, unrealistic completion times). The final sample included 447 respondents (Mage = 33.21; SDage = 8.39; female = 230; see supplementary file 5 for demographic information across all studies), taking on average 9.52 min (SD = 4.13 min) to complete the CMBQ.

Data Analysis

Participants completed the 22-item CMBQ on one occasion. To bolster scale validity, item order was randomised differently from the EFA study. Data were screened for outliers (standardised z values > 3.29), and outliers were Winsorized (n = 18 from 9834 cases = .18%). Data were subjected to CFA using Structural Equation Modelling (SEM) in SPSS AMOS version 25 in which a three-factor, and then a two-factor model was tested using both correlated-factor and bifactor models (four CFAs in total). Bifactor models were tested to indicate whether the CMBQ assesses S-R generation and C–M change on a single factor, on two or three separate correlated factors, or both. Bifactor models allow the identification of a general factor in addition to multiple unique factors (Laguna et al., 2019) which are of interest in psychometric questionnaire research (Chen et al., 2016). See supplementary file 4 for details concerning the CFA, including the testing of bifactor models, in the current study. To compare the correlated three-factor and two-factor models, we conducted a χ² difference test, but because of limitations in using χ² for large sample sizes (Yuan & Bentler, 2004), we inspected fit indices for each model to determine which model had the stronger model fit (Table II) with CFI differences of <.01 indicating no difference (Kline, 2005).

Table II. Model fit indices for the correlated and bifactor, three-factor and two-factor models.

Model	χ^2	df	CFI	TLI	NFI	RMSEA (90% CI)	Δχ^2
1. Correlated three-factor	233.344	113	.96	.96	.93	.05 (.040-.058)
2. Correlated two-factor	169.441	86	.97	.97	.95	.05 (.036-.057)	Model 1 vs. 2: 67.11, df = 29, p < .01
3. Bifactor three-factor	201.412	99	.97	.96	.94	.05 (.039-.058)
4. Bifactor two-factor	146.070	74	.98	.97	.96	05 (.035-.058)	Model 3 vs. 4: 55.34, df = 25, p < .05

Note. All models were statistically significant at the p < .001 level. χ2 = chi-square test statistic, RMSEA = Root Mean Square Error of Approximation, CI = Confidence Interval, CFI = Comparative Fit Index, TLI = Tucker-Lewis Index. The bolded model demonstrated the best fit in the correlated and bifactor models.

Results

The 22-item correlated three-factor model was not an acceptable fit, χ² = 461.559, df = 202, p < .001, RMSEA = .05 (90% CI = .047–.060), CFI = .93, NFI = .89, TLI = .92. After the removal of five items with low factor loadings (e.g., Hair et al., 2009), the correlated three-factor model was an acceptable fit, χ² = 233.344, df = 113, p < .001, RMSEA = .05 (90% CI = .040–.058), CFI = .96, NFI = .93, TLI = .96. The 22-item correlated two-factor model was not an acceptable fit, χ² = 461.141, df = 202, p < .001, RMSEA = .05 (90% CI = .047–.060), CFI = .93, NFI = .89, TLI = .92. After the removal of seven items with low factor loadings (including the same items that were removed for the three-factor model), the correlated two-factor model was an acceptable fit, χ² = 169.441, df = 86, p < .001, RMSEA = .05 (90% CI = .036–.057), CFI = .97, NFI = .95, TLI = .97. Comparing the acceptable three-factor and two-factor models, a χ² difference test (Δχ² = 63.903, df = 27, p = .01) and CFI differences (ΔCFI = .01) indicated that the two-factor model offers a superior model fit. The acceptable correlated three-factor and two-factor models were taken forward to bifactor analyses. For bifactor analyses results see supplementary file 4 For fit indices comparisons see Table II, and for correlated and bifactor two-factor item loadings, Relative Parameter Bias, and IECV see Table III.

Table III. Correlated and bifactor two-factor item loadings, Relative Parameter Bias (RPB), and item explained common variances (IECV).

				Bifactor
Item number		Uni factor loading	RPB	EICV	General factor loading	Specific factor loading
	S-R
10	How I feel is completely dictated by the things that happen to me in my life.	.631	−0.412	.507	.447	.441
40	My feelings are entirely determined by peoples’ actions towards me.	.602	−3.459	.032	.135	.748
50	My feelings are completely controlled by the situation I am in.	.676	−0.519	.441	.445	.501
76	My emotions are entirely caused by what people do around me.	.625	−3.223	.036	.148	.769
17	My emotions are caused entirely by others’ actions towards me.	.634	−3.227	.036	.150	.781
55	My emotions are caused entirely by the things that happen to me.	.763	−0.255	.622	.608	.474
35	What happens to me entirely dictates how I feel.	.819	−0.365	.542	.600	.552
19	My emotions are completely dictated by what happens to me.	.861	−0.305	.578	.660	.564
	C-M
44	To change how I feel, my thoughts about the situation need to change.	.565	−9.273	.009	.055	.563
45	To change how I feel, I need to change what I think about things around me.	.558	−8.964	.010	.056	.568
20	Thinking differently about the situation will change how I feel.	.753	−7.461	.014	.089	.747
56	To change how I feel, I can change my thoughts about the situation.	.770	−5.937	.021	.111	.763
23	I can change my emotions by changing how I think about the situation.	.787	−9.781	.009	.073	.786
75	Because I can choose to think differently, I can choose to feel differently about the situation.	.686	−14.244	.004	.045	.683
69	To control my emotions, I need to change the way I think.	.727	−14.468	.004	.047	.733

Study 3: Second CFA, and Criterion Validity

The purpose of study 2 was to test the factor structure of the CMBQ using CFA with a separate sample of adults. The CFA supported a correlated two-factor structure, comprising 15-items (S-R generation = 8; C–M change = 7). In study 3, we recruited a separate adult sample to firstly confirm the two-factor structure within a new sample, and secondly to conduct criterion validity tests on the CMBQ. Convergent validity was examined in relation to measures of thought control ability, and emotion regulation. If the CMBQ assesses C–M change and S-R generation beliefs in the hypothesised manner, scores reflecting greater C–M change beliefs and lower S-R generation beliefs should be related to greater thought control ability, and better emotion regulation (cognitive reappraisal). Concurrent validity was examined in relation to measures of mental health, and emotion reactivity. Scores reflecting greater C–M change beliefs and lower S-R generation beliefs were hypothesised to be related to greater mental health, and lower emotion reactivity. Finally, we determined floor (lowest possible score) and ceiling (highest possible score) effects as an additional marker of psychometric quality.

Participants

In a separate sample to the previous studies, 432 participants completed a battery of questionnaires including the CMBQ. Item order was again randomised to limit the extent to which scale validity could be attributed to item order. Prolific was used to recruit an adult sample following the same criteria and procedures as was used for the EFA (study 1) and CFA (study 2) studies. The blacklist facility in Prolific ensured that participants who had participated in the EFA and CFA studies could not take part in this study. A total of 17 respondents’ data were excluded due to poor data quality (i.e., straight line responding, unrealistic completion time, incomplete responses). The final sample comprised 415 respondents (Mage = 33.61; SDage = 8.75; female = 265; see supplementary file 5 for demographic information across all studies).

Procedure

All 415 participants completed the 15-item CMBQ, the Emotion Regulation Questionnaire (ERQ; Gross & John, 2003), and the Positive Mental Health (PMH) scale (Lukat et al., 2016), taking on average 11.03 min (SD = 5.17 min) to complete. A subset of 214 participant also completed the Emotion Reactivity Scale (ERS; Nock et al., 2008), the Affective Reactivity Index (ARI; Stringaris et al., 2012), and the Thought Control Ability Questionnaire (TCAQ; Luciano et al., 2005), taking on average 7.41 min (SD = 3.37 min) to complete. The ERQ (cognitive reappraisal) and TCAQ indicated thought and emotion regulation, and the ERS and ARI indicate emotion reactivity.

Measures

Cognitive Mediation Beliefs. The 15-item CMBQ (S-R generation = 8 items, C–M change = 7 items) was scored on a 1 (strongly disagree) to 5 (strongly agree) Likert-scale. Cronbach’s α for the current sample was .90 for S-R generation, and .81 for C–M change. S-R generation and C–M change were negatively correlated (r = -.20, p < .001).

Emotion Regulation. The ERQ is a 10-item self-report measure that assesses respondents’ tendency to regulate their emotions in two ways: (1) Cognitive Reappraisal and (2) Expressive Suppression (Gross & John, 2003). In the current sample, Cronbach’s α was .89 for Cognitive Reappraisal, and .82 for Expressive Suppression. Only cognitive reappraisal was used from the ERQ, due to its theoretical relationship with cognitive mediation.

Thought Control Ability. For the current study, the 20-item version of the TCAQ (Williams et al., 2010) was utilised (see Feliu-Soler et al., 2019, for a review). The TCAQ assesses the perceived ability to control unwanted thoughts (e.g., “I often cannot avoid having upsetting thoughts”). In the current sample, Cronbach’s α was .92.

Emotion Reactivity. The ERS (Nock et al., 2008) is a 21-item measure of individuals’ experience of emotion reactivity, indicating the extent to which an individual experiences emotions (a) in response to a wide array of stimuli (emotion sensitivity), (b) strongly or intensely (emotion intensity), and (c) for a prolonged period of time before returning to baseline level of arousal (emotion persistence). Cronbach’s α was .92 for emotion sensitivity, .88 emotion intensity, and .89 emotion persistence.

Affective Reactivity. The ARI was designed with large studies in mind and is therefore concise (Stringaris et al., 2012). For the current study, the 6-item (irritability questions) version of the ARI was used for brevity. The ARI measures chronic irritability across (a) threshold for an angry reaction; (b) frequency of angry feelings/behaviors; (c) duration of such feelings/behaviors. In the current sample, Cronbach’s α was .84.

Positive Mental Health. The 9-item PMH scale (Lukat et al., 2016) assesses emotional aspects of well-being reflecting a single holistic concept of positive emotionality related to positive mental health. Cronbach’s α was .90 in the current study.

Data Analysis

Data were screened for outliers (standardised z values > 3.29), and outliers were Winsorized (n = 26 from 34,030 cases = .08%). Data were Winsorized for CMBQ cases (n = 18), ERQ cases (n = 2), and ARI cases (n = 7). First, the 15-items of the CMBQ (S-R generation = 8 items; C–M change = 7 items) were subjected to CFA using SEM in AMOS version 25, in which a correlated two-factor model was tested. The same goodness of fit indices posited by Schermelleh-Engel et al. (2003) were used to determine acceptable fit, and the same MI guidelines by Rossier et al. (2012) were used. Then, following similar research (e.g., Groarke & Hogan, 2018), convergent and concurrent validity was tested by examining Pearson’s bivariate correlation co-efficients (using SPSS version 25) between the CMBQ subscale scores, and TCAQ and ERQ (cognitive reappraisal) scores (convergent), and ARI, ERS, and PMH scores (concurrent). It was hypothesised that greater C–M change and lesser S-R generation scores would be significantly related (p < .05) to greater TCAQ, ERQ (cognitive reappraisal), and PMH scores, and lower ARI and ERS scores. For concurrent validity testing we also conducted three hierarchical regression analyses, one for each outcome (PMH, ARI, and total ERS). As sex and age have been found to influence emotion regulation in past research (sex, Goubet & Chrysikou, 2019; age, Ortega, 2009), for all regression analyses age and sex were entered into step 1. The two CMBQ subscales of C–M change and S-R generation were entered into step 2.

Results

CFA

The 15-item two-factor model was a good fit, χ² = 219.878, df = 86, p < .001, RMSEA = .06 (90% CI = .041–.071), CFI = .95, NFI = .92, TLI = .94. See Table IV for factor loadings.

Table IV. Item properties, internal consistency, inter-item correlations, and descriptives, of the 15-item CMBQ in study 3.

						Inter-item correlation
Item number		β	R^2	α	M(SD)	Range	M(SD)
	S-R			.90
10	How I feel is completely dictated by the things that happen to me in my life.	.70	.49		3.09(1.01)	.378-.521	.462(.061)
40	My feelings are entirely determined by peoples’ actions towards me.	.77	.60		2.64(.94)	.383-.601	.498(.084)
50	My feelings are completely controlled by the situation I am in.	.68	.47		2.83(.97)	.416-.547	.496(.049)
76	My emotions are entirely caused by what people do around me.	.83	.69		2.67(.96)	.446-.627	.515(.072)
17	My emotions are caused entirely by others’ actions towards me.	.73	.53		2.57(.96)	.378-.627	.510(.093)
55	My emotions are caused entirely by the things that happen to me.	.84	.71		2.95(1.00)	.405-.665	.521(.090)
35	What happens to me entirely dictates how I feel.	.59	.35		2.98(1.01)	.484-.730	.557(.087)
19	My emotions are completely dictated by what happens to me.	.72	.52		2.82(.98)	.492-.730	.578(.086)
	C-M			.81
44	To change how I feel, my thoughts about the situation need to change.	.40	.16		3.87(.78)	.379-.521	.440(.051)
45	To change how I feel, I need to change what I think about things around me.	.54	.29		3.79(.73)	.349-521	.436(.069)
20	Thinking differently about the situation will change how I feel.	.80	.63		3.82(.81)	.421-617	.515(.075)
56	To change how I feel, I can change my thoughts about the situation.	.66	.44		3.73(.78)	.384-.643	.514(.095)
23	I can change my emotions by changing how I think about the situation.	.76	.58		3.73(.78)	.401-.643	.536(.100)
75	Because I can choose to think differently, I can choose to feel differently about the situation.	.68	.46		3.61(.90)	.349-.571	.467(.095)
69	To control my emotions, I need to change the way I think.	.34	.12		3.82(.78)	.423-.558	.498(.048)

Note. Items 35 and 19 (S-R) had high (>.70) maximum inter item correlations, so should be investigated further in a different sample to indicate whether they are surplus to requirements or not.

Criterion Validity

Convergent validity. In the Pearson’s correlation analysis (Table V), S-R generation scores were negatively associated with cognitive reappraisal (p < .001), and thought control ability (p < .001), and C–M change scores were positively associated with cognitive reappraisal (p < .001), and thought control ability (p < .001).

Table V. Pearson’s correlation coefficients.

	M(SD)	1	2	3	4	5	6	7	8	9
1. S-R	2.87(.78)	-
2. C-M	3.80(.52)	-.20**	-
3. ERQ cognitive reappraisal	29.40(6.37)	-.26**	.58**	-
4. TCAQ	58.33(14.49)	-.43**	.39**	.46**	-
5. PMH	17.47(5.54)	-.26**	.37**	.55**	.65**	-
6. ARI	2.77(2.77)	.37**	-.14*	-.24**	-.49**	-.46**	-
7. ERS sensitivity	13.69(9.54)	.50**	-.26**	-.35**	-.74**	-.56**	.64**	-
8. ERS intensity	10.00(6.68)	.42**	-.18**	-.24**	-.67**	-.47**	.51**	.87**	-
9. ERS persistence	6.12(4.35)	.47**	-.29**	-.35**	-.78**	-.57**	.54**	.85**	.77**
10. ERS total	29.81(19.51)	.49**	-.25**	-.33**	-.77**	-.56**	.61**	.98**	.94**	.90**

Note. **p < .001, *p < .01

Concurrent validity. The Pearson’s correlation analysis (Table V) revealed small to large associations between CMBQ subscale scores, and the ERS subscales. Greater C–M change scores and lower S-R generation scores were related to less emotion sensitivity, intensity, and persistence. In the linear hierarchical regression analyses, across all models step 1 (age and sex) did not explain a significant proportion of variance (R²Δ < .01, p > .05). In step 2 and in separate models (all p < 001), the two CMBQ subscale scores explained 17% of variance in PMH (C–M change: β = .34, t = 7.32, p < .001; S-R generation: β = -.20, t = 4.25, p < .001), 13% of variance in the ARI (C–M change: β = -.07, t = 1.08, p > .05; S-R generation: β = .35, t = 5.40, p < .001), and 26% of variance in the ERS (C–M change: β = -.16, t = 2.71, p < .01; S-R generation: β = .45, t = 7.47, p < .001).

Construct Validity

Floor and ceiling effects. No more than 15% of a sample should obtain the top or bottom score (McHorney & Tarlov, 1995). In the current sample, no participants scored the lowest (1), and two participants (0.5%) scored the highest (5) possible score for S-R generation, and for C–M change no participant scored the lowest (1), and five participants (1.2%) scored the highest (5) possible score.

Discussion

The results of study 3 confirmed the correlated two-factor structure of the 15-item CMBQ. Also, greater C–M change scores and lesser S-R generation scores were positively related to cognitive reappraisal and thought control ability, demonstrating convergent validity. Greater C–M change scores and lesser S-R generation scores were positively related to positive mental health, and negatively related to affective and emotion reactivity, demonstrating concurrent validity. Finally, descriptive statistics revealed acceptable floor and ceiling effects, which indicates some construct validity.

Study 4: Test-Retest Reliability

Results thus far preliminarily indicate that the CMBQ has sufficient construct, concurrent, and convergent validity. In study 4 we examined the test-retest reliability of the CMBQ. Test-retest reliability is important as it indicates the reproducibility of a measure (Law, 2004). The CMBQ reflects an individual’s general beliefs about cognitive mediation and is thus theoretcially not situationally or temporally dependent. As such, we hypothesised that CMBQ scores would remain stable across a seven-day interval. In addition, we examined the extent to which scores on the CMBQ were subject to social desirability by including the brief social desirability scale (BSDS; Haghighat, 2007) at time 1. If the CMBQ is subject to social desirability, subscale scores should be related to BSDS scores in a correlation analysis. As such, we hypothesised that CMBQ subscale scores would be unrelated to BSDS scores.

Participants

Guidelines indicate a sample size of at least 50 participants at two time-points to detect an estimated intra-class coefficients (ICC) of .80 with a 95% confidence interval (CI) from .70 to .90 (Giraudeau & Mary, 2001). Law (2004) suggest aiming for ICCs of at least 0.75 with lower limits of CIs above .60, and an overall sample size of at least 30 participants. Based on these works, and on the sampling guidelines of Polit (2014), we recruited 134 participants to complete the CMBQ at two separate timepoints, seven-days apart. We anticipated more than 50% attrition (Polit, 2014) and therefore anticipated approximately 67 participants being retained at timepoint 2. Prolific was used to recruit an adult sample following the same criteria and procedures as was used for the previous studies in this paper. At time 1, we used the blacklist facility in Prolific to ensure that participants who had participated in the previous studies could not take part in this study. At time 2, we used the whitelist facility in Prolific to ensure only participants who completed time 1 were approached to complete time 2. The final sample was 134 participants at time 1 (Mage = 33.60; SDage = 8.37; female = 96; see supplementary file 5 for demographic information across all studies), and 87 participants (65%) were retained at time 2.

Procedure

All 134 participants completed the 15-item CMBQ and the BSDS (Haghighat, 2007) at time 1. Test-retest intervals of 1 week or 2 weeks are typical and consistent with guidance from the literature (Polit, 2014), therefore all participants were approached exactly seven days after time 1 to complete the CMBQ at time 2. The 15-items of the CMBQ were randomised at time 1 so that the order of items differed from the previous studies. At time 1, participants took 5.17 min (SD = 2.84) on average to complete the questionnaires, and at time 2, participants took 4.55 min (SD = 3.44) on average to complete the questionnaires

Measures

Cognitive Mediation Beliefs. The 15-item CMBQ was completed at time 1 and time 2. In the current sample, Cronbach’s α at time 1 for S-R generation (M = 2.84, SD = .86) was .93, and .88 for C–M change (M = 3.77, SD = .61). At time 2, Cronbach’s α for S-R generation (M = 2.69, SD = .81) was .93, and .83 for C–M change (M = 3.82, SD = .53).

Social desirability. The 4-item BSDS (Haghighat, 2007) was developed as a pragmatic assessment of social desirability. The BSDS provides a total score, and higher scores represent a greater likelihood of providing socially desirable self-report responses. BSDS was completed at time 1 (M = 5.97, SD = 1.21), where the sample size was maximal.

Data Analysis

Data were screened for outliers (standardised z values > 3.29), and outliers were Winsorized (n = 8 from 5177 cases = .15%). Data were Winsorized for CMBQ cases at time 1 (n = 5), and time 2 (n = 3). To assess the test-retest reliability of the CMBQ, the ICC and associated 95% CIs (e.g., Koo & Li, 2016) were calculated between time 1 and time 2 data, using two-way mixed effects, single measurement, absolute agreement, parameters. Koo and Li (2016) suggest that ICC and CI ranges between 0.5 and 0.75 indicate moderate reliability, values between 0.75 and 0.9 indicate good reliability, and values greater than 0.90 indicate excellent reliability. Also, Pearson’s correlation coefficient between time 1 and time 2 (Law, 2004) of .70 indicates test–retest reliability (Barker et al., 2002). We also calculated the Pearson’s correlation coefficient between the CMBQ scores, and the BSDS.

Results

CMBQ Test-Retest Reliability

For S-R generation, we found an ICC of .79 (CI, .70-.86), F(1,86) = 3.54, p > .05. In addition, a correlation coefficient I of .80 (p < .001) was found between time 1 and time 2. For C–M change, we found an ICC of .66 (CI, .52-.76), F(1,86) = .114, p > .05. A correlation coefficient(r) of .66 (p < .001) was found between time 1 and time 2. Taken together, the CMBQ demonstrated moderate-good test-retest reliability.

CMB and Social Desirability

There was no significant association between BSDS scores and S-R (r = -.12, p > .05), and C–M (r = -.01, p > .05) scores.

Discussion

The results of study 4 demonstrated stability of CMBQ scores across a seven-day interval and indicated that responses were not subject to social desirability.

Study 5: Validity Within a Mental Health Population

Thus far in the current paper we have recruited healthy adult samples to test the validity and reliability of the CMBQ. In study 5 we recruited a separate adult sample of participants who have a known (diagnosed) mental health condition (MHC) in order to test the validity of the CMBQ in a non-healthy population. In order to achieve this, we conducted CFA to confirm the correlated two-factor structure of the CMBQ, and regression analyses to assess the concurrent validity of the CMBQ in the same way as we did in study 3, using PMH and ERS scores as self-report indicators of mental health and emotion reactivity respectively. We hypothesised that greater C–M change scores and lower S-R generation scores would be related to greater PMH and lower ERS scores, as was revealed in study 3.

Methods

Participants

In a separate sample (n = 228) to the previous studies, participants (Mage = 34.79; SD age = 9.59; female = 135; see supplementary file 5 for demographic information across all studies) who have been diagnosed with a current and ongoing mental health condition (MHC), as indicated in their Prolific.ac profile, were recruited. Of the 228 participants, 186 (81.58%) had received CBT in the past, and 195 (85.53%) had taken medication for their MHC in the past. Participants completed a battery of questionnaires including the CMBQ, the PMH scale, and the ERS. The samples size was determined by the criteria for CFA (a ratio of 10 participants per item; Kyriazos, 2018). Participants were recruited via Prolific, so that we could recruit a sample by the same procedures as was used for all previous studies in the current paper, via the blacklist facility. On average, participants took 8.24 min (SD = 6.37) to complete the questionnaires.

Measures

Cognitive Mediation Beliefs. The 15-item CMBQ demonstrated Cronbach’s α of .89 for S-R generation (M = 2.95, SD = .79), and .88 for C–M change (M = 3.65, SD = .72). Based on goodness of fit indices posited by Schermelleh-Engel et al. (2003), the CFA demonstrated that the correlated two-factor model was a good fit, χ² = 124.571, df = 84, p = .003, RMSEA = .05 (90% CI = .028–.063), CFI = .98, NFI = .94, TLI = .97.

Positive Mental Health. In the current study, the PMH scale (Lukat et al., 2016) was used (M = 21.19, SD = 5.48), for which the Cronbach's α was .89.

Emotion Reactivity. For the ERS (Nock et al., 2008) the Cronbach’s α was .90 for emotion sensitivity (M = 31.73, SD = 9.36), .89 for emotion intensity (M = 23.00, SD = 6.70),.83 for emotion persistence (M = 14.11, SD = 3.84), and .96 for total ERS (M = 68.84, SD = 18.68).

Data Analysis

Prior to main analyses, data were screened for outliers (standardised z values > 3.29), and outliers were Winsorized (n = 2 from 10,260 cases = .02%). Data were Winsorized for CMBQ cases only. We conducted hierarchical multiple regression analyses to examine the relationships between CMBQ scores and PMH scores, and total ERS scores (concurrent validity). For regression analyses, age and sex were entered into step 1, and the two CMBQ subscale scores were entered into step 2.

Results

For PMH, step 1 (age and sex) did not explain a significant proportion of variance (R²Δ = -.01, p > .05). In step 2, CMBQ scores explained 17% of variance in PMH (p < .001). In the final model, R²Δ = .15, F(4, 223) = 11.32, p < .001, C–M change beliefs were positively related to PMH (β = .39, t = 6.19, p < 001). S-R generation beliefs were not significantly related to PMH (β = -.10, t = 1.62, p = .107). In sum, those with greater C–M change scores reported greater PMH scores.

For total ERS, step 1 (age and sex) explained 9% of variance (p < .001) in ERS. In step 2, CMBQ scores explained 13% of variance in ERS (p < .001). In the final model, R²Δ = .20, F(4, 223) = 15.52, p < .001, age was negatively related to ERS (β = -.15, t = 2.39, p = .018) and sex was positively related to ERS (β = .19, t = 3.13, p = .002). S-R generation beliefs were positively related to ERS (β = .29, t = 4.76, p < .001), and C–M change beliefs were negatively related to ERS (β = -.18, t = 2.94, p = 004). In sum, older, male participants, who reported greater C–M change scores and lower S-R generation scores, reported lower ERS scores.

Discussion

In study 5 we found evidence that the factor structure and internal validity of the CMBQ is acceptable for individuals with a diagnosed MHC. We also found that the concurrent validity revealed in study 3 was partially supported in study 5. That is, CMBQ scores explained a significant proportion of variance in both PMH and ERS scores, but S-R generation beliefs were not significantly related to PMH scores. The results indicate that the CMBQ may be suitable for use in MHC populations, but more expansive research needs to be undertaken before we suggest that the CMBQ could be used unilaterally with MHC samples. We did not assess the effects of previous treatment on outcomes, due to the large inequality within the binary data assessing whether participants had received previous CBT or medication.¹ Future research could ensure equal sample sizes and test whether and to what extent past treatment for a MHC influences CMBQ scores, and or, whether the effects of past treatment on emotion reactivity and mental health is mediated by CMBQ scores.

General Discussion

The 15-item CMBQ assesses two superordinate emotion beliefs; S-R generation beliefs (emotions are caused by events) and C–M change beliefs (changes in cognition lead to emotion change). The factor structure of the CMBQ was the result of EFA (study 1) and CFA (study 2), confirmed in additional CFAs (studies 3 and 5), demonstrating concurrent (study 3 and 5) and convergent validity (study 3), and test-retest reliability (study 4). Greater C–M change beliefs and lower S-R generation beliefs were related to higher emotion regulation tendencies, a greater ability to control their thoughts, more positive mental health, and lower emotion reactivity. One notable outcome of the factor analyses was that the four conceptualised emotion beliefs only partially bore fruit. We generated items to assess S-R generation, S-R change, C–M generation, and C–M change emotion beliefs, but through EFA analyses and subsequent CFA analyses, only S-R generation and C–M change emotion beliefs formed reliable factors. We approached the analyses in a rigorous way following guidelines for EFA and CFA, and therefore the loss of potential factors is part and parcel of the criteria used for fit indices and factor loadings. It is unclear why S-R change and C–M generation did not form suitable factors, but the rigorous approach we took lends confidence to the veracity of the two emergent factors (S-R generation and C–M change). The two factors that did emerge were related to measures of cognitive reappraisal, thought controllability, and perhaps more importantly, emotion reactivity and positive mental health. Thus, the emergent S-R generation and C–M change factors are potentially valuable for understanding the types of emotion beliefs that may be antecedent or related to emotion regulation and emotion.

Especially noteworthy about the results of the present paper is that participants’ naïve emotion beliefs (i.e., presumably uninformed academically about emotion regulation theory and research) are in sync with what the extant literature indicates for effective emotion regulation. That is, research demonstrates that cognitive change is an effective emotion regulation strategy (e.g., McRae et al., 2010), and in the current paper greater endorsement of C–M change beliefs related to more adaptive emotion regulation and emotion outcomes. Relatedly, S-R generation beliefs were deleterious to emotion regulation and emotion outcomes, which is in line with the proposed shortcomings of an S-R viewpoint of emotion (e.g., Lazarus, 1999). Thus, when people’s emotion beliefs align with emotion regulation theory and research evidence, more adaptive outcomes are observed.

The CMBQ extends the research concerning emotion beliefs by proposing two emotion beliefs that are not captured by current conceptualisations; goodness and controllability (e.g., Ford & Gross, 2019). Findings of the current paper indicate support for postulations that individual beliefs concerning how emotions arise and how they can be managed may precede emotion regulation attempts (Trincas et al., 2016). Findings are also consistent with Ford and Gross' (2018) assertion that emotion beliefs are deeply consequential for emotion and emotion regulation. The current paper also reinforces the importance of examining emotion regulation using a two-factor approach with distinct generation and change (management) beliefs (Cole et al., 2004). Indeed, we find that C–M change and S-R generation beliefs do not directly oppose one another, evidenced by their small to moderate negative relationship. Whilst greater C–M change and lower S-R generation beliefs appear to be advantageous for emotion regulation, it is clear that those with higher C–M change beliefs do not necessarily have parallel low S-R generation beliefs. Indeed, although clearly different conceptually, Means for C–M change and S-R generation beliefs in each study are not polar opposites (e.g., study 3 C–M change Mean = 3.80, S-R generation = Mean 2.87). It is possible for a person to have high C–M change beliefs and high S-R generation beliefs; just because a person believes emotions emerge as a direct result of the situation, does not prohibit them from believing in, and attempting, cognitive change.

However, there is some evidence that cognitive change is more effective in decreasing negative affect when emotions are generated in a top-down, rather than a bottom-up, fashion (McRae et al., 2012). Bottom-up emotion generation refers to the elicitation of emotion by the presentation of a stimulus that is inherently emotional (McRae et al., 2012; Otto et al., 2014), whilst top-down emotion regulation refers to the elicitation of emotion by the activation of high-level appraisals (Ochsner et al., 2009). Bottom-up emotion generation is a stimulus-focused view of emotional processing (McRae et al., 2012), a notion characterised by S-R generation beliefs in the present study, and reliably elicits amygdala activity (Zald, 2003). Top-down emotion generation is a cognition-focused view of emotional processing (McRae et al., 2012), characterised by C–M generation beliefs in the present study (which did not form a reliable factor). Importantly, bottom-up emotions are elicited largely by perceptions, which can be automatic and need not be accessible to conscious awareness (e.g., Phelps & LeDoux, 2005), whereas top-down emotions elicited largely by cognitions are usually accessible to conscious awareness (McRae et al., 2012).

Therefore, there is a potential disjunction between S-R generation beliefs and C–M change beliefs, in that emotions that are seen to arise via S-R may not bring forth cognitive-based efforts to regulate the emotion. As such, a belief in S-R generation may discount C–M change as a legitimate method of emotion management. Indeed, the differences in psychological and neural mechanisms for bottom-up vs. top-down emotion generation (e.g., Otto et al., 2014) may have important consequences for emotion regulation attempts. Top-down emotion generation elicits a pattern of prefrontal activation similar to that of cognitive change (Otto et al., 2014). Because top-down generation activates the appraisal, and cognitive change manipulates it, cognitive change may be more effective when performed upon top-down generated emotions (McRae et al., 2012). A person can have high C–M change beliefs and high S-R generation beliefs, but subsequent emotion regulation may suffer from this seemingly contradictory belief pairing. Future research should determine the extent to which S-R generation and C–M change beliefs are orthogonal, and the extent to which they may interact to predict emotion and emotion regulation. A discrepancy value (C–M change minus S-R generation), much like that of the hedonic balance score derived from the positive and negative affect dimensions of the PANAS (e.g., Allen et al., 2017), could be considered. Perhaps, it is the extent to which C–M change beliefs predominate over S-R generation beliefs, that is most important for emotion regulation.

The current paper has some practical implications. High C–M change beliefs, and low S-R generation beliefs, represent a volitional viewpoint of emotion and emotion regulation, reflecting the notion of individual emotional responsibility. The message here is not that, when an individual experiences an unpleasant or unwanted emotion, that it is all their fault. There are of course situational events that strongly influence our emotions, but even in situations where the event is highly evocative and stressful (e.g., bottom-up), it could be argued that perceiving that emotion change can take place by changing the way one thinks, provides some tangible volition over emotion. Interventions that target cognitive change (e.g., second wave CBTs), should first encourage C–M change beliefs, and discourage S-R generation beliefs. Practitioners could help the people they work with to first understand that they can have a strong influence on their emotions, by managing their thoughts about life events, prior to teaching cognitive change techniques such as cognitive reappraisal.

Limitations and Future Research

The CMBQ assesses respondents’ beliefs in relation to experiencing unpleasant or unwanted emotions, but does not assess the precise situation that may have given rise to the emotion, and does not compel the respondent to change the emotion. This consideration is important, because emotions are generated when an attended to situation is interpreted as being central to one’s personal, social, and cultural goals (Scherer et al., 2001). The C–M and S-R beliefs captured by the CMBQ do not reflect value judgements about the attended to situation, and the respondent is not asked whether or not they think the emotion should be changed. The CMBQ captures superordinate and dispositional emotion beliefs, to which respondents can apply their own meaning with regards to personal, social, and cultural goals. Whilst superordinate and dispositional factors are important for emotional regulation (e.g., Hermann et al., 2014), the CMBQ could be adapted to particular situations (e.g., social-evaluative) and emotions (e.g., anxiety), in order to examine the influence of personal, social, and cultural goals on CMBQ scores and consequent emotion outcomes.

One of the reasons why high C–M change beliefs could be adaptive is because it makes cognitive change more likely. But cognitive change is not the only option for emotion regulation. Aside from the situational selection and modification, attentional deployment, and response modulation strategies highlighted in the process model (Gross, 2014), people can also employ acceptance strategies, characterised by non-elaborative, non-judgmental, presence-centred awareness in which thoughts, feelings, and sensations are accepted as they occur (e.g., Baer et al., 2004). Acceptance strategies have been shown to reduce subjective distress (Hofmann et al., 2009) and associated psychophysiological responses (Wolgast et al., 2011), and enhance positive emotion (Dan-Glauser & Gross, 2015).

Acceptance and cognitive change could be comparable in terms of their regulatory effectiveness (e.g., reduce behavioural avoidance; Wolgast et al., 2011). Although, Boehme et al. (2019) recently found that cognitive change was most effective concerning subjective emotional experience (less arousal and unpleasantness), sympathetic downregulation, and dampening of attentional capture as reflected in altered subsequent information processing. However, acceptance led to comparable effects in skin conductance response. Boehme et al. (2019) conclude that cognitive change is predominant over acceptance, but that acceptance might be a useful strategy in contexts where cognitive change is not possible. Given the strong support for acceptance and commitment therapy (ACT; Gloster et al., 2020), future research could develop and examine acceptance and or mindfulness emotion beliefs alongside S-R generation and C–M change beliefs to determine potential complementarity.

Researchers should conduct additional tests through replication studies that include additional CFA and criterion validity tests. This could include investigating potential C–M generation and S-R change beliefs, which did not form reliable factors in the EFA and CFA stages of the current paper. Furthermore, predictive validity studies should be undertaken using longitudinal designs to study the extent to which the CMBQ predicts emotion regulation and mental health outcomes over time. In addition, future researchers should build on the findings of study 5 to test the CMBQ across different samples such as adolescent, athletic, student, and pathological samples. Relatedly, the participants represented in the current paper do not demonstrate diverse characteristics. Specifically, Whites were overrepresented in the current paper, making up 89.9% of the sample which is 3.9% above the population statistics (86%) for the U.K (Office of National Statistics, 2019). There was an underrepresentation of Asian (3.1% to the 7.5% national statistics), Black (1.8% to the 3.3% national statistics), and Mixed ethic groups (1.8% to the 2.2% national statistics). This imbalance could be a function of the recruitment facility we used (Prolific.ac).

The current paper has some strengths that should enhance the veracity of the findings. The samples sizes for each study were suitable for the analyses; important against a backdrop of under-recruitment in the area of psychometric development (Costello & Osborne, 2005). In addition, we recruited separate samples for each study in order to limit the attribution of results to cohort specific factors. This included separate samples for EFA and CFA stages. In each sample, internal reliability of the CMBQ was acceptable, thus increasing confidence in the measurement structure. Lastly, we include a rigorous multi-stage item-development, review, and refinement process, ensuring construct and face validity.

Conclusion

The initial validity and reliability tests concerning the CMBQ reveal a correlated two-factor structure indicating S-R generation and C–M change beliefs. Results suggest that beliefs reflecting higher C–M change beliefs and lower S-R generation beliefs relate to greater self-reported mental health and lower emotion reactivity, and that CMBQ scores remain stable over a seven-day interval. Additional research is required to further test the validity and reliability of the CMBQ, especially longitudinally and cross-culturally.

Acknowledgments

The authors would like to thank the expert and industry panel for their time and diligence in reviewing the items generated for the CMBQ. Thank you also to the novice review panels, especially Jamie Hinkin.

The authors have complied with APA ethical standards in the treatment of the sample.

Supplemental data

Supplemental data for this article can be accessed https://doi.org/10.1080/10503307.2020.1871524.

Notes

1 We ran the analyses inclusive of CBT and pharmaceutical treatment history entered at step 2; this did not alter the results markedly. The reported effects occurred whilst controlling for whether participants had received CBT and or pharmaceutical treatment in the past.