Chronic and contextual identity salience: Assessing dual-dimensional salience with the Identity Salience Questionnaire (ISQ)

ABSTRACT

Identity Salience is a common construct within social identity research. However, researchers note that it is poorly defined and inconsistently operationalized. We posit that identity salience comprises two elements: chronic (perpetually thinking about the identity) and contextual (only thinking about the identity when prompted) salience. We present evidence for this claim through the development and validation of the dual-dimensional Identity Salience Questionnaire (ISQ). Studies 1-2 (Ns=414; 1,069) provide exploratory and confirmatory factor analytic evidence of the ISQ among LGBTIQA+ participants. Study 2 also provides evidence of measurement invariance, convergent, predictive, and discriminant validity, and internal reliability. Study 3 (N=318) indicates strong test-retest reliability. Study 4 (N=107 social psychologists) confirms the ISQ’s content validity. Future research for the ISQ is discussed.

KEYWORDS:

• Social identity
• identity salience
• LGBTIQA+
• measurement
• confirmatory factor analysis

Well-established and influential frameworks in identity scholarship, such as Identity Theory (Stryker, 1968; Stryker & Serpe, 1982), Social Identity Theory (Tajfel & Turner, 1979), and Self-Categorisation Theory (Turner, 1985; Turner et al., 1987), provide important insights for how individuals evaluate their sense of self (i.e., their self-concept) according to the social groups they belong to, and the environmental contexts with which these social groups hold relevance. Extant research supports the notion that the process of identifying with such groups is largely driven by moment-to-moment situational inputs (i.e., the environmental factors that may trigger the self-awareness of the identity, see e.g., Hogg et al., 1995; Stets & Burke, 2000). Whilst these environmental inputs can shape the relevance and awareness of an individual’s social group memberships (to varying degrees), individuals can also vary in their degrees to which their social identities are considered perpetually relevant to their sense of self, at any given moment (Ashmore et al., 2004; Leach et al., 2008).

This sense of relevance and awareness is broadly known as identity salience – a construct we believe is ill-defined, lacks consistent operationalizations, and is often conflated with other dimensions and processes of social identification. The goal of this paper is to therefore enhance the definitional clarity on what identity salience is, and to then provide evidence of a new way of measuring identity salience that accurately reflects this definition. We start by reviewing existing definitions, and subsequently propose a new definition of identity salience as a dual-dimensional construct (comprised of elements relating to both its contextual and chronic nature). We then highlight several issues with the operationalization and measurement of identity salience across the literature, before concluding with a series of studies that provide evidence pertaining to the development and validation of a new identity salience measure that assesses this dual-dimensional conceptualization – the Identity Salience Questionnaire (ISQ).

Identity salience: Existing and proposed conceptualisations

Identity salience has been defined in multiple ways in previous research. As derived from Identity Theory, early definitions of identity salience reflect the probability or likelihood that an identity will be activated or invoked, and subsequently “performed,” within a particular context or situation (Stryker, 1968). This definition operationalizes identity salience as a behavioral phenomenon that is driven by momentary contextual factors within the individual’s environment, whereby an identity is considered to be salient when it is performed, or otherwise enacted, according to the current context (e.g., the disclosure of a parental identity to others or the act of performing tasks in accordance with a workplace-oriented identity, see Brenner et al., 2014, 2023; Burke, 2023; Markowski & Serpe, 2021; Stryker & Serpe, 1982; Thoits, 2020). Stemming from this, McCall and Simmons (1978) suggest that an individual’s social or “role” identities are organized in a “cognitive hierarchy” with the most prominent (or central) identities being at the top of the list and are therefore primed to be more readily activated (and subsequently “performed”) in any given context. That is, McCall and Simmons (1978) assert that the salience of an identity is defined by its activation and cognitive self-awareness, and suggest that for some individuals, this activation might be relatively enduring (i.e., regardless of their fleeting environmental contexts).

According to social psychological perspectives (e.g., Social Identity and Self-Categorisation Theories: Tajfel & Turner, 1979; Turner, 1985; Turner et al., 1987), identity salience has been described as the chronic accessibility of an identity’s relevance to the individual, especially among those with stronger ties to their social groups. This definition aligns with McCall and Simmons (1978) suggestion that the degree of an identity’s relevance and self-awareness extends beyond the dependency of situational cues and inputs. Put otherwise, individuals might be chronically aware of their social identities, rather than only being aware of them if they are brought into the perceiver’s attention from cues within their environments.

A key issue for conceptualizing salience solely by its chronic relevance, however, is how this proposition conflates identity salience with the social identification construct of identity centrality (i.e., the degree of importance an individual places on their identity, relative to their self-concept; Rosenberg, 1979; see also Ashmore et al., 2004, for a review of the differing terminologies of identity centrality). As noted by several others (e.g., Ashmore et al., 2004; Brenner et al., 2014, 2023; Hinton et al., 2022; Hogg et al., 1995; Stryker & Burke, 2000; Stryker & Serpe, 1994), the terms centrality and salience are frequently, yet incorrectly, used interchangeably with one another to broadly capture this degree of identity importance. Recently, researchers have responded to these noted conceptual inconsistencies with strong evidence showing that the constructs of identity centrality (defined as the level of importance placed upon an identity) and identity salience (defined as the chronic degree of awareness pertaining to the identity) are conceptually distinct (Begeny & Huo, 2017; Quinn et al., 2014; Utku & Sayılan, 2023). Yet, current research lacks valid and psychometrically robust measurement to (a) fully capture the construct of salience, and (b) adequately distinguish it from identity centrality, as defined in these ways.

To help future research understand the nuance of the centrality-salience distinction, we must first establish a clear definition of identity salience, before developing appropriate measurement to capture this definition. The current paper expands upon this latter “chronic awareness” definition of salience to further acknowledge that the nature of identity salience can also be driven by situational contexts (as noted in earlier definitions; Ashmore et al., 2004; Brenner et al., 2023; Stryker, 1968; Stryker & Serpe, 1982; Thoits, 2020). Hence, we propose that identity salience is dual-dimensional, and should be defined as: the degree of perpetual (chronic) or situationally dependent (contextual) activation and self-awareness of an individual’s identity group membership(s).

Issues in measurement

While some studies have experimentally explored how identities can be made salient (i.e., through priming situational cues or contexts), and the outcomes associated with this (see e.g., Haslam et al., 1999), much less attention has been paid to adequately measuring identity salience as the general degree of identity self-awareness. The definitional inconsistencies described thus far have conceptualized and/or operationalized identity salience as either (a) a contextually driven sense of activation, (b) the chronic degree of identity self-awareness and relevance, or (c) a construct synonymous with identity centrality. These inconsistencies have subsequently led to various issues in the measurement of identity salience (Brenner et al., 2023; Burke, 2023).

One such example pertains to operationalizing identity salience by its contextual definitions, where it has been subsequently measured by the likelihood of identity disclosure or “performance” in a particular context (e.g., at work), or by examining with whom an individual will mention their identity (Brenner et al., 2014, 2023; Burke, 2023; Markowski & Serpe, 2021; Stryker & Serpe, 1982). This approach may be suitable to some role-based identities (e.g., parent), yet limited in its applicability to other identities (e.g., those that are marginalized). For instance, among those with stigmatized identities who do not wish to disclose their identity for various reasons (e.g., for protection against potential stigma or biased treatment), non-disclosure may not equate to these identities being less salient (or brought into awareness/activated). Indeed, concealing an identity has the potential to make this identity more salient, particularly if the concealment is effortful and intentional. Resultantly, these individuals might inaccurately underreport their levels of identity salience if it were to be operationalized as a behavioral process of disclosure. Despite these limitations, however, this behavioral approach provides important insights into the role of context (Brenner et al., 2023; Burke, 2023; Forehand et al., 2002) – a factor that other measures of identity salience have neglected.

Another identity salience measurement issue occurs when this construct is conflated with identity centrality. For instance, several researchers claim to be measuring “identity salience,” but have used items that solely reflect the degree of importance placed on an identity (e.g., “How important is your [X] identity to you?” in Haslam et al., 1999; see also Gonzalez et al., 2018; Ramirez & Galupo, 2019; Scroggs & Vennum, 2021; Woodford et al., 2015). To further compound this issue, some existing measures include items of both identity salience (e.g., as the chronic frequency of thought) and identity importance (e.g., how central an identity is to the sense of self) within the same factor, and collectively conceptualize this factor as identity centrality (Cameron, 2004; Leach et al., 2008).

An interesting consequence of conflating identity importance and salience (e.g., Leach et al., 2008) has been that researchers seem to have accepted that these constructs are interchangeable. It is also possible that some researchers have not been aware that these measures conflate these (potentially distinct) constructs (e.g., Brenner et al., 2014, 2023; Hinton et al., 2022; Markowski & Serpe, 2021; Stryker & Serpe, 1994; Thoits, 2020). In support of this, recent research has found that identity importance and identity salience differentially relate to various psychosocial outcomes (e.g., mental ill-health and discrimination experiences; Begeny & Huo, 2017; Quinn et al., 2014; Utku & Sayılan, 2023), highlighting the need for their differentiation. Despite these advancements to conceptual knowledge in social identity scholarship, the scarce body of evidence that has differentiated identity salience from identity importance does so by using non-validated and self-developed single (e.g., Quinn & Chaudoir, 2009) or multi- (e.g., Begeny & Huo, 2017; Quinn et al., 2014; Utku & Sayılan, 2023) item scales that solely ask respondents to rate their levels of chronic identity self-awareness (or perpetual degree of thought; see also Cameron, 2004; Leach et al., 2008). Put simply, operationalizing and measuring identity salience solely as chronic awareness can be beneficial by helping separate this construct from identity importance (or centrality), but as noted previously, the role of context may also be an important factor when shaping the understanding of identity salience (Brenner et al., 2023; Burke, 2023).

The current research

As discussed, existing scholarship defines (and subsequently operationalizes) identity salience in various ways. An underlying theme that appears consistent across these conceptualizations relates to the cognitive activation or self-awareness of an identity – whether this activation is prompted by the situation at hand (i.e., contextually driven) or is more perpetually enduring across contexts (i.e., chronically accessible). Whilst the conceptual issues pertaining to identity salience are certainly not new, and have indeed been the topic of discussion and debate among several researchers (e.g., Ashmore et al., 2004; Brenner et al., 2023; Hinton et al., 2022; Stryker & Serpe, 1994; Thoits, 2020), to our knowledge, no existing measure of identity salience captures both its chronic and contextual elements of identity self-awareness, akin to our proposed definition stated earlier. It is our view that this is a weakness of the literature, and that identity salience should be operationalized and measured in a way that is conducive to this proposed dual-dimensional conceptualization.

In response to these current conceptual inconsistencies, this paper presents the development and validation of a new identity salience measure that captures elements of both chronic and contextual identity self-awareness. We do this with data from individuals who identify as Lesbian, Gay, Bisexual, Transgender/Gender-Diverse, Intersex, Queer, Asexual, or with another (LGBTIQA+) sexual minority or gender-diverse identity. LGBTIQA+ individuals are often subjected to identity-relevant stigma (Hill et al., 2020; Meyer, 2003) and are a minority group with unique identity experiences (e.g., the ability to conceal the identity; Pachankis et al., 2020). Further, insights from a recent literature review found that sub-groups within the LGBTIQA+ community also differ in their levels of identity centrality (Hinton et al., 2022), and are likely to have varying degrees of self-awareness pertaining to their identity group memberships (Begeny & Huo, 2017; Utku & Sayılan, 2023). Thus, we believe examining identity salience within this group will be of value.

Psychometric and validation hypotheses

In order to test the primary aim of the paper, several hypotheses were generated (see Table 1). Firstly, we expected the new identity salience items would yield strong factor loadings (>.70) across both exploratory (H1) and confirmatory (H2a – H2b) analyses. Next, the two-factor (chronic and contextual) structure of our proposed scale (H2a) would be a good fit for the data, as indicated by model fit indices within recommended cutoff ranges (i.e., CFI ≥ .95; TLI ≥ .95; SRMR ≤ .08; RMSEA ≤ .06; Hu & Bentler, 1999; Kline, 2016). Further, given that our sample consists of multiple identity groups (i.e., LGBTIQA+ sub-groups), we expected that the confirmed structure would show stability across groups (H2b) determined by nil differences in model fit (ΔCFI’s ≤ .01; Cheung & Rensvold, 2002) at each level of model constraints.

Table 1. Summary of hypotheses tested across studies.

	Hypotheses	Study Tested in
Scale Development Hypotheses:
H1: Exploratory	At least one factor with strong loadings will emerge when exploring the factor structure of our proposed identity salience scale consisting of items reflecting both chronic and contextual salience.	Study 1
H2a: Confirmatory	The exploratory factor structure found in H1 will be confirmed, again yielding strong factor loadings and an adequate model fit.	Study 2
H2b: Confirmatory (Measurement Invariance)	We expect that our retained and confirmed model of identity salience would remain stable across different LGBTIQA+ identity groups.	Study 2
Validity Hypotheses:
H3: Convergent	Identity salience will strongly correlate with other measures of identity centrality, especially those which include items of identity salience.	Study 2
H4: Predictive	Identity salience (particularly items that reflect contextual salience) will correlate with identity concealment.	Study 2
H5: Discriminant	Identity salience will not correlate (or correlate weakly) with measures of mental health and well-being.	Study 2
H6: Content	The assessment of our proposed identity salience items will be rated by social psychologists as adequate reflections of the construct of identity salience.	Study 4
Reliability Hypotheses:
H7: Internal Consistency	Identity salience will have excellent estimates of internal consistency.	Study 1–3
H8: Test-Retest	Identity salience will have strong test-retest reliability (i.e., temporal stability).	Study 3

Once the factor structure was confirmed, we assessed several forms of validity (H3–H6), as recommended by Boateng et al. (2018). To our knowledge, no existing scale of identity salience exists that solely contains items reflecting the degree of thought or awareness relating to an identity. However, as our items were broadly constructed from relevant (salience) items that were embedded within existing scales of identity centrality, to test for convergent validity (H3), we predicted strong correlations between each identity salience factor and measures of identity centrality, specifically measures that use items pertaining to the frequency in which an identity is thought of (e.g., Cameron, 2004; Leach et al., 2008). To test the predictive validity (H4) of our new scale, we predicted a small-to-moderate relationship between identity salience and concealment. Two reasons underpinned this prediction. First, concealment is negatively related to identity centrality (including scales with embedded identity salience items) among LGBTIQA+ individuals (Hinton et al., 2022; Le Forestier et al., 2022). Second, the behavioral act of concealing an identity is largely context-dependent and overlaps with the above-mentioned behavioral definitions of identity salience (i.e., degree of disclosure; Brenner et al., 2023; Burke, 2023). Thus, we predicted that identity concealment would be associated with identity salience, specifically through its contextual elements. For discriminant validity (H5), we expected that identity salience would not be related to (or be weakly related to) indicators of mental health (depression, anxiety, stress) and well-being (self-esteem, life satisfaction). We based this prediction on the findings from Begeny and Huo (2017) indicating that identity salience (as a separate construct from identity importance/centrality) was unrelated to symptoms of mental ill-health (see also Utku & Sayılan, 2023). The final assessment of validity pertained to the content of the identity salience items themselves (H6). When social psychology researchers (particularly those with knowledge of identity scholarship) were presented with our scale items, we expected the majority would agree that these items were an adequate reflection of identity salience (as we have conceptualized it here). Finally, we expected that our identity salience scale would show both strong internal consistency (H7: determined by α’s ≥ .80) and test-retest reliability (H8: determined by Intraclass Correlation Coefficients [ICC’s] ≥ .80).

Data availability

All data from Studies 1–4 are available on the open science framework (https://osf.io/ghqyx/), including R code and variable codebooks. No studies were pre-registered.

Study 1

Study 1 explores the factor structure (H1) of the items proposed for the Identity Salience Questionnaire (ISQ) and gathers initial evidence for the internal consistency (H7) of the emerging factors within a sample of gay men.

Method

Participants

As is common with Exploratory Factor Analyses (EFA), sample size was considered based on a 20:1 ratio of participants to scale items (Kline, 2016). Thus, to test the factor structure of a scale with nine items, a minimum sample size of 180 was needed. Although this ratio can be prone to error rates, the primary purpose of EFA is to simply explore covariance and factor structures amongst items, rather than provide confirmatory evidence (see Osborne, 2014). With this consideration in mind, we oversampled to allow for maximum power.

Five-hundred and seventy-seven participants responded to our online survey recruiting gay men. At the data cleaning stage, 49 participants who initially responded to our advertisement were excluded from analyses as they did not consent to the use of their data. A further 77 participants were excluded as they did not provide responses for any of the ISQ items. Finally, 37 participants were excluded for identifying their gender as non-male and/or their sexuality as non-gay, leaving a final homogenous sample of 414 gay (cisgender) men included in this study. Participants were aged between 18–73 years old (M = 31.57, SD = 11.18), and were mostly born in either the U.S.A. (n = 295; 71.3%) or the UK (n = 85; 20.5%), with few participants born in other countries.

Measures and procedure

Gay men were recruited to take part in our online survey via the Prolific platform. After consenting, participants completed a brief series of demographic questions, followed by the nine ISQ items. Participants then completed a series of experimental tasks and other questionnaires as part of a larger study (not described here). The new salience items are described below.

Identity salience item generation

Utilizing a deductive approach to item generation (see Boateng et al., 2018), we first reviewed the existing literature of identity salience and other relevant identity processes (see Hinton et al., 2022), as well relevant existing measures to assess the items that other researchers have used (e.g., Begeny & Huo, 2017; Cameron, 2004; Hughes & Hurtado, 2018; Leach et al., 2008; Quinn & Chaudoir, 2009; Quinn et al., 2014; Sellers et al., 1998). After synthesizing this evidence, the lead author then generated an initial pool of items that were expected to closely match the construct of identity salience as presented here (i.e., items operationalized by the [chronic or contextual] degree to which an individual may think about, or is otherwise aware of, their identity, to reflect the conceptualization of an identity being activated or brought into awareness). These items were then reviewed (and re-reviewed) by all authors and other colleagues at the authors’ institutions for relevance.

As we aimed to develop a brief measure of identity salience (the ISQ), a final pool of nine items (see Table 2) were selected from the larger pool based on how well they reflected how individuals think about their identity in either chronically or contextually relevant ways. The development of these items reflected previous literature that solely operationalized identity salience as the (chronic) frequency of thought (Begeny & Huo, 2017; Hughes & Hurtado, 2018; Quinn & Chaudoir, 2009; Quinn et al., 2014), however, to align with the duality of our proposed conceptualization, we expanded this by including items that also reflect the degree to which an identity is thought of only when contextually relevant. Participants were asked to respond to these items (in a randomized order) on a scale from 1 (strongly disagree) to 7 (strongly agree). Items reflected the salience of their “gay” identity.

Table 2. Item-Level Descriptive Statistics, inter-Item correlations, and pattern matrix factor loadings (EFA; study 1).

		Inter-Item Correlations								Model 1 (8-items) Loadings		Model 2 (6-items) Loadings
	M (SD)	1.	2.	3.	4.	5.	6.	7.	8.	Factor 1	Factor 2	Factor 1	Factor 2
Item 1: My [X] identity is often at the forefront of my mind	3.82 (1.76)	-								−.96	.05	.83	−.03
Item 2^a: Being [X] is mostly all that I think about	2.43 (1.52)	.65***	-							−.74	.10	.80	.09
Item 3: I am often thinking about my [X] identity, despite what situation I am in	3.24 (1.60)	.69***	.62***	-						−.75	−.07	.79	−.10
Item 4: I only think about my [X] identity if someone or something has mentioned it	4.03 (1.59)	−.31***	−.16***	−.32***	-					.01	.88	−.04	.86
Item 5: My [X] identity is only at the forefront of my mind when someone brings it up	4.05 (1.65)	−.21***	−.16***	−.31***	.75***	-				−.04	.87	.00	.87
Item 6: I only think about my [X] identity when prompted	3.65 (1.58)	−.22***	−.12*	−.25***	.69***	.69***	-			−.05	.83	.02	.81
Item 7: I am always aware of my [X] identity	4.98 (1.58)	.64***	.37***	.48***	−.30***	−.22***	−.24***	-		−.61	−.09	-	-
Item 8: My [X] identity rarely crosses my mind	3.79 (1.66)	−.44***	−.27***	−.34***	.49***	.43***	.48***	−.44***	-	.32	.44	-	-
Item 9: I find that I am aware of my [X] identity when it is mentioned in my environment	5.17 (1.33)	.08	−.03	.07	.09	.22***	.07	.07	−.07	-	-	-	-

*p < .05, **p < .01, ***p < .001. ^aThe means and standard deviations are reported on the untransformed Item 2 variable for ease of interpretation. Correlations and factor loadings for Item 2 were conducted on the log-transformed corrected variable. For Model 1 (with 8-items): KMO = .818, Bartlett’s test of sphericity p < .001, cumulative eigenvalues for both factors = 70.5% of variance. For Model 2 (the final model with 6-items): KMO = .761, Bartlett’s test of sphericity p < .001, cumulative eigenvalues for both factors = 79.5% of variance. Factor loading in bold indicate items that were retained (i.e., loadings ≥ .70) within factors.

Results

Exploratory factor analysis (EFA)

Examination of item-level skewness indicated that item 2 was positively skewed, and subsequently corrected by logarithmic transformation. All other items were normally distributed. Bivariate correlations were then explored amongst our proposed nine items to assess suitability for EFA and found that item 9 showed no correlations with other items, except for one, and was subsequently excluded from further analyses (see Table 2).

The remaining eight items were subjected to an EFA conducted in SPSS (version 27) using principal axis factoring with oblique (direct oblimin) rotation. Given the briefness of this proposed scale, we opted for stringent criterion of only retaining items that had strong factor loadings (i.e., ≥ .70, equivalent to items sharing at least 50% of their variance with factors; Kline, 2016). Items 7 and 8 did not meet this threshold and were excluded from further analyses (see Model 1 in Table 2 for loadings).

A final EFA was conducted with the remaining six items, again using principal axis factoring and direct oblimin rotation. An assessment of the Kaiser-Meyer-Olkin (KMO) measure, Bartlett’s test of sphericity, and the anti-image correlation matrix (all items ≥ .73) indicated that sampling adequacy was sufficient (see Table 2). Upon inspection of the scree plot and the factor eigenvalues, results from this EFA reveal a two-factor solution accounting for a total of 79.5% of variance (Factor 1: Eigenvalue = 3.09, 51.5% of variance; Factor 2: Eigenvalue = 1.68, 28.0% of variance). A two-factor solution was also supported by a parallel analysis, in which our obtained eigenvalues exceeded those simulated for a dataset with 6 indicators and 414 participants (Factor 1 Eigenvalue = 1.16; Factor 2 Eigenvalue = 1.09; Patil et al., 2017). The final loadings were strong for both Factor 1 (i.e., “Chronic salience;” ≥ .79) and Factor 2 (i.e., “Contextual salience;” ≥ .81). All item-level descriptive statistics, correlations, and factor loadings are presented in Table 2. The finalized items of the ISQ are reported in Appendix.

Factor correlations and reliability

Factors scores were computed by obtaining the average across items within factors to allow for correlation analyses. Both the chronic salience (M = 3.17, SD = 1.43, α = .85) and contextual salience (M = 3.91, SD = 1.44, α = .88) factors had excellent estimates of internal consistency. Chronic salience was also moderately, and negatively, correlated with contextual salience, r(412) = −.28, 95% CI [−.36, −.19], p < .001. That is, the more participants chronically think about their gay identity, the less they think about their gay identity only when prompted by environmental cues.

Discussion

Study 1 provided initial evidence of a two-factor scale that captures both the chronic and contextual elements of identity salience. The sub-scales were negatively correlated and had excellent internal consistency. The factor psychometrics and evidence of validity are yet to be confirmed.

Study 2

Study 2 aimed to confirm the factor structure (H2a) emerging from Study 1 by submitting items to a Confirmatory Factor Analysis (CFA). This study also assessed if the proposed factor structure holds consistently across different LGBTIQA+ identity groups (H2b). Finally, Study 2 explored the validity of the ISQ by assessing its relationship with other variables (H3–H5) and estimates of internal consistency (H7).

Method

Sample size consideration and data cleaning

To ensure we had an adequate sample size to conduct CFAs for LGBTIQA+ sub-groups, we used the guidelines presented by Wolf et al. (2013) who conducted multiple Monte Carlo simulations to determine the minimum sample size needed to obtain true effects with an estimated power of 80% and α = .05. Based on the factor loadings obtained from Study 1 (i.e., ~.80), a two-factor CFA with three items per factor (and high loadings) needs a minimum sample size of 120. Further, as the relationships between identity salience and multiple variables are of interest in this study, we conducted an a priori power analysis in G*Power (Faul et al., 2009). Using the smallest effect size of interest (i.e., the meta-analytic correlation between centrality and concealment, r = −.16; Hinton et al., 2022), with 1-β = .80, and α = .05, a minimum sample of 304 participants is needed. As we aimed to assess our model across multiple identity groups, and considering survey dropout rates, we oversampled to ensure our study had adequate power.

In total, 1,637 participants responded to our online survey, however several exclusions were made for the following reasons: 54 participants provided duplicate responses, 15 did not provide consent for their data to be used, 8 participants were under 18 years old, 14 participants did not identify as LGBTIQA+, and 477 participants had large amounts of missing data (i.e., they had only responded to one or two demographic items after consenting).

Participants

A final sample of 1,069 LGBTIQA+ participants were included. Of these participants, 939 provided full responses to the survey, with the remaining 130 providing partial responses (listwise deletion was applied to all analyses, and thus df and n values vary between analyses). Participants were aged between 18–95 years old (M = 35.78, SD = 14.57), with almost all (n = 1,062; 99.3%) residing in Australia at the time of survey completion. Participants responded to an open-text response question assessing ethnicity. This was subsequently re-coded by response frequency and indicated that the vast majority of participants identified as White (n = 885; 82.8%, collapsed across Anglo-Celtic/White Australian and European backgrounds), followed by East/South-east Asian or Pacific Islanders (n = 75; 7.0%), multi-ethnic identities (n = 34; 3.2%), Aboriginal, Torres Strait Islander, or Māori identities (n = 31; 2.9%), with the remaining participants identifying with other ethnicities (n = 24; 2.2%) or providing unclassifiable responses (n = 20; 1.9%).

Prior to commencing, participants were instructed to state whether they best identified as either cisgender, gender-diverse (e.g., transgender, non-binary), or intersex (i.e., born with variations to their sex characteristics). They were then prompted to select the gender and the sexuality that they best identified with from a list. One of these identities was used as a reference category to answer subsequent questions. In order to achieve a range of identity groups, we made the a priori decision to classify identities hierarchically. As such, intersex participants (regardless of their sexuality or gender identity) responded to items about their intersex identity; gender-diverse participants (regardless of their sexual orientation) responded to items about their gender identity; and cisgender participants responded to items about their sexuality identity. Table 3 outlines the target identities that were included in this study and age statistics for each identity group. As some identity groups were smaller than others, we aggregated these into six LGBTIQA+ groups (see Table 4) that are commonly reported in gender and sexuality literature to allow multi-group analyses (Lyons et al., 2020).

Table 3. Descriptive statistics for target LGBTIQA+ identity groups for study 2 and study 3.

	Study 2 (N = 1,069)		Study 3 (N = 318)		Age
	n	%	n	%	Range	M (SD)
Cisgender Participants, Sexuality Identity Target:
Gay	269	25.2	89	28.0	18–95	43.41 (14.79)
Lesbian	133	12.4	37	11.6	18–75	37.65 (15.11)
Bisexual	193	18.1	74	23.3	18–70	28.47 (10.02)
Pansexual	41	3.8	8	2.5	18–66	33.85 (14.71)
Queer	52	4.9	14	4.4	18–68	33.65 (12.87)
Asexual/Aromantic	28	2.6	9	2.8	18–47	26.50 (8.15)
Demisexual	5	0.5	2	0.6	22–30	25.60 (3.36)
Sexually Diverse	8	0.7	0	0.0	18–53	34.63 (13.56)
Gender-Diverse Participants, Gender Identity Target:
Gender-Diverse (Male)	24	2.2	6	1.9	18–76	44.83 (17.54)
Gender-Diverse (Female)	33	3.1	8	2.5	20–68	44.61 (13.08)
Gender-Diverse	20	1.9	2	0.6	19–63	37.15 (14.27)
Transgender	17	1.6	0	0.0	20–61	40.00 (14.35)
Trans-Male	24	2.2	7	2.2	19–61	30.92 (10.59)
Trans-Masculine	26	2.4	8	2.5	18–43	25.88 (7.51)
Trans-Female	23	2.2	3	0.9	18–67	42.83 (15.03)
Trans-Feminine	16	1.5	4	1.3	18–77	34.63 (16.64)
Non-Binary	76	7.1	29	9.1	18–60	30.07 (11.67)
Gender Queer	24	2.2	3	0.9	18–67	31.13 (12.56)
Agender	21	2.0	8	2.5	19–50	27.43 (7.63)
Gender Fluid	27	2.5	6	1.9	18–63	30.33 (11.45)
Intersex Participants, Intersex Identity Target:
Intersex	9	0.8	1	0.3	37–67	48.67 (11.18)

Table 4. Aggregated LGBTIQA+ identity groups for analyses.

Identity Groups	n
Gay	258
Includes only cisgender males who identify as gay.
Lesbian	144
Includes cisgender females who identify as either lesbian or gay.
Bisexual	193
Includes cisgender participants who identify as bisexual.
Other Sexuality	134
Includes cisgender participants who identify as either pansexual, queer, asexual/aromantic, demisexual, or with another sexuality identity.
Transgender (Binary)	163
Includes gender-diverse participants who identify as either gender-diverse (male), gender-diverse (female), transgender, trans-male, trans-masculine, trans-female, or trans-feminine (i.e., those with binary [e.g., male/female] gender-diverse identities)
Gender-Diverse (Non-Binary and Fluid)	168
Includes gender-diverse participants who identify as either gender-diverse, non-binary, gender queer, agender, or gender fluid (i.e., those with gender identities that fall outside of, or more fluidly within, the binary identities of male and female)

Note. Given that intersex identities are neither sexuality nor gender identities, we did not include intersex participants (n = 9) in our aggregated identity groups.

Materials and procedure

Participants were recruited for this online survey either through the Prolific platform (n = 213) or through a mix of social media posts to various LGBTIQA+ Australian groups or paid Facebook advertisements (n = 856). Participants were instructed to provide online consent and were then prompted to complete mental health and well-being measures (in a randomized order). Participants then answered a series of demographic questions and were finally asked to respond to a battery of questionnaires that assessed various aspects of their targeted LGBTIQA+ identity.¹

Identity salience

The 6-items from the ISQ were administered as per Study 1. In this study, the relevant (target) identity (see Table 3) for each participant was inserted to each ISQ item (to replace [X]). Items were averaged to create a factor score to allow correlational analyses. Both the chronic salience (α = .84) and contextual salience (α = .89) sub-scales showed excellent internal reliability in the current sample.

Identity centrality measures

To explore the convergent validity of the ISQ, we included four measures of identity centrality. We selected two measures of identity centrality where the items solely reflected the importance of the individual’s identity, and another two measures that included items which reference both the importance to the individuals’ self-concept and the frequency in which that identity is thought of (i.e., salience). The two measures that explored identity centrality as importance to the self were the 5-item centrality sub-scale of the Lesbian, Gay, and Bisexual Identity Scale (LGBIS; sample item: “I believe being [X] is an important part of me;” Mohr & Kendra, 2011) and the 4-item identity importance sub-scale of the Collective Self-Esteem Scale (CSES; sample item: “The [X identity group] I belong to is an important reflection of who I am;” Luhtanen & Crocker, 1992). Both sub-scales from the LGBIS (α = .86) and the CSES (α = .85) showed excellent reliability in the current sample.

The first scale that included both importance (e.g., “In general, being [X] is an important part of my self-image”) and salience (e.g., “The fact that I am [X] rarely enters my mind”) items in its measure of identity centrality was the 7-item centrality sub-scale of the Three Factor Model of Social Identity (TFMSI; Cameron, 2004). The second measure that included items relating to both importance (e.g., “The fact that I am [X] is an important part of my identity”) and salience (e.g., “I often think about the fact that I am [X]”) was the 3-item centrality sub-scale of the Multicomponent Model of In-Group Identification (MMII; Leach et al., 2008). Both sub-scales from the TFMSI (α = .91) and the MMII (α = .84) showed excellent reliability in the current sample.

Finally, and in line with Begeny and Huo (2017), the centrality sub-scale of the MMII was also split into two sub-dimensions of identity importance (i.e., the two items from Leach et al., 2008 that reference importance; r = .84, p < .001) and identity salience (i.e., the one item from Leach et al., 2008 that references salience, plus an additional item developed by Begeny & Huo, 2017: “In a lot of situations, I find myself thinking about the fact that I am [X];” r = .84, p < .001). Except for the LGBIS sub-scale, which was responded to on a scale of 1–6 with the same anchor labels listed below, all other identity centrality sub-scales were responded to on a scale from 1 (strongly disagree) to 7 (strongly agree). Sub-scales were mean scored, with higher scores indicating greater levels of centrality (importance and/or salience). In each sub-scale, “[X]” refers to the participants’ target identity (Table 3).

Identity concealment

To assess the predictive validity of the ISQ, participants completed the 3-item concealment motivation sub-scale of the LGBIS (Mohr & Kendra, 2011). Items (e.g., “My sexual orientation² is a very personal and private matter”) were responded to on a scale of 1 (strongly disagree) to 6 (strongly agree), were mean scored, and were coded such that higher scores indicate greater levels of identity concealment. This sub-scale indicated excellent internal reliability in the current sample (α = .90).

Mental health and well-being measures

To assess the discriminant validity of the ISQ, we asked participants to respond to a variety of measures assessing general mental health symptomology and psychological well-being (i.e., without reference to their target identity group). Depression, anxiety, and stress measures were employed to measure mental health symptomology. Depression was measured using the 10-item Centre for Epidemiologic Studies Short Depression Scale (CES-D; Andresen et al., 1994. Participants were asked to reflect on scale items (e.g., “I was bothered by things that usually don’t bother me”) within the past week and responded on a scale from 0 (rarely/none of the time) to 3 (all of the time). Anxiety was measured using the 7-item Generalised Anxiety Disorder Questionnaire (GAD-7; Spitzer et al., 2006). Participants were asked to reflect on scale items (e.g., “Feeling nervous, anxious or on edge”) within the last 2 weeks and responded on a scale from 0 (not at all) to 3 (nearly every day). Finally, stress was measured using the 10-item Perceived Stress Scale (PSS; Cohen et al., 1983). Participants were asked to reflect on how often the scale items (e.g., “ … been upset because of something that happened unexpectedly?”) occurred within the last month and responded on a scale from 0 (never) to 4 (very often). All measures of mental health symptomology showed excellent reliability in the current sample (CES-D α = .88; GAD-7 α = .92; PSS α = .91) and were mean scored, with higher scores indicating greater endorsement of symptoms (i.e., worse mental health).

Finally, psychological well-being was assessed with two measures: Life satisfaction and self-esteem. Life satisfaction was measured using the 5-item Satisfaction with Life Scale (SWLS; Diener et al., 1985). Items (e.g., “In most ways my life is close to my ideal”) were responded to on a scale from 1 (strongly disagree) to 7 (strongly agree). Self-esteem was measured using the 10-item Rosenberg Self-Esteem Scale (RSES; Rosenberg, 1965). Items (e.g., “On the whole, I am satisfied with myself”) were scored on a response scale from 1 (strongly disagree) to 4 (strongly agree). Both well-being measures showed excellent reliability in the current sample (SWLS α = .89; RSES α = .92) and were also mean scored, with higher scores indicating greater levels of well-being (i.e., more positive well-being).

Results

Confirmatory factor analysis (CFA)

Prior to conducting analyses, the distributions of the six ISQ items were explored. Again, item 2 was positively skewed and therefore corrected by logarithmic transformation. All other items were normally distributed. All ISQ items from the full LGBTIQA+ sample were subjected to CFA using the lavaan package in R (Rosseel, 2012), with items 1–3 loading onto the chronic salience factor, and items 4–6 loading onto the contextual salience factor. CFA was conducted using robust maximum likelihood estimation (i.e., with robust Satorra-Bentler scaled estimations) and with the marker method of model identification (i.e., constraining first item loadings per factor to 1). Results from this CFA revealed strong factor loadings (≥.76; see Figure 1) and acceptable model fit: χ²(8) = 66.96, p < .001, CFI = .98, TLI = .97, RMSEA = .08, 90% CI [.07, .10],³ SRMR = .04.

Figure 1. CFA Model of the ISQ factors (chronic and contextual salience) indicating item loadings, factor correlations, and item residual variance (study 2; N = 1,035).

Notes. CFA = Confirmatory Factor Analysis. ISQ = Identity Salience Questionnaire.

Given that the factors were strongly and negatively correlated (Figure 1), we also conducted an alternative CFA model in which all six items loaded onto a single factor (i.e., to explore the alternative that the contextual salience items might just be inversed items of the chronic salience factor, and vice versa). This alternative model showed poor fit to the data: χ²(9) = 702.74, p < .001, CFI = .79, TLI = .64, RMSEA = .27, 90% CI [.26, .29], SRMR = .15. Results from a χ² difference test further confirm that this alternative model was significantly weaker in fit compared to the two-factor model (Δχ²(1) = 387.72, p < .001), thus the two-factor model was retained.

Measurement invariance

We also sought to explore the stability of model fit for this two-factor model across LGBTIQA+ identity groups. We used the six aggregated LGBTIQA+ identity groups outlined in Table 4 to perform a multi-group CFA. Of note, all six identity groups had an adequate sample size for this analysis to be performed (e.g., n’s > 120; Wolf et al., 2013). Multiple CFAs were first conducted within each identity group separately to assess model fit (see Table 5). With the exception of the Gender-Diverse (Non-binary and Fluid) identity group, which yielded slightly weaker model fit across some indices (e.g., RMSEA = .11), all other LGBTIQA+ sub-groups indicated acceptable model fit.

Table 5. Model fit indices, standardized factor loadings, and factor correlations (r), for each LGBTIQA+ sub-sample (study 2).

	N	χ^2	CFI	TLI	RMSEA [90% CI]	SRMR	Chronic Salience (Loadings)			Contextual Salience (Loadings)			rfactors
							Item 1	Item 2	Item 3	Item 4	Item 5	Item 6
Identity Group:
Gay	249	χ^2(8) = 15.31, p = .053	.99	.98	.06 [.01, .10]	.04	.80	.81	.77	.85	.87	.78	−.50
Lesbian	141	χ^2(8) = 12.78, p = .120	.99	.98	.07 [.00, .12]	.05	.82	.69	.78	.93	.85	.80	−.63
Bisexual	191	χ^2(8) = 21.63, p = .006	.98	.96	.09 [.05, .14]	.05	.86	.74	.81	.94	.80	.85	−.63
Other Sexuality	129	χ^2(8) = 16.23, p = .039	.98	.96	.09 [.02, .15]	.05	.87	.82	.73	.87	.86	.84	−.55
Transgender (Binary)	155	χ^2(8) = 13.41, p = .099	.99	.98	.07 [.00, .12]	.05	.83	.81	.85	.94	.79	.80	−.55
Gender-Diverse (Non-Binary and Fluid)	162	χ^2(8) = 24.88, p = .002	.96	.92	.11 [.07, .17]	.06	.91	.67	.77	.91	.84	.76	−.53

We then explored configural invariance of our model by exploring its performance across the six identity groups, without any model constraints applied. The results from this multi-group CFA indicate that configural invariance was achieved, with the model showing acceptable fit: χ²(48) = 101.35, p < .001, CFI = .98, TLI = .97, RMSEA = .08, 90% CI [.06, .10], SRMR = .04. Next, we explored metric invariance by employing the same multi-group CFA with all loadings constrained to be equal across groups. Results indicate that metric invariance was also achieved, with the model showing acceptable fit: χ²(68) = 124.14, p < .001, CFI = .98, TLI = .98, RMSEA = .07, 90% CI [.05, .09], SRMR = .05. Finally, we assessed scalar invariance by conducting the multi-group CFA, but with all loadings and intercepts constrained to be equal across groups. Again, scalar invariance was achieved by the model showing excellent fit to the data: χ²(88) = 147.82, p < .001, CFI = .98, TLI = .98, RMSEA = .06, 90% CI [.05, .08], SRMR = .06. An assessment of the difference between configural CFI and metric CFI (ΔCFI < .01), and between metric CFI and scalar CFI (ΔCFI < .01), were both within the acceptable range for confirming that measurement invariance was achieved across the six LGBTIQA+ identity groups.

ISQ validity

Prior to conducting our correlational analyses, variables were checked for skewness and were all found to be normally distributed. Table 6 outlines the descriptive statistics and correlations between variables for the full LGBTIQA+ sample. Again, we found a moderate-strong and negative correlation between the ISQ sub-scales, r(1033) = −.49, 95% CI [−.53, −.44], p < .001. Correlations between these sub-scales and the measures used to assess convergent, predictive, and discriminant validity are presented below. In addition, supplementary materials (Tables S1–S6) also explore these correlations, separated by LGBTIQA+ identity groups.

Table 6. Descriptive statistics and zero-order correlations between ISQ sub-scales and measures of validity (study 2).

		M (SD)	1.	2.	3.	4.	5.	6.	7.	8.	9.	10.	11.	12.	13.
	1. ISQ (Chronic)	3.32 (1.43)	-
	2. ISQ (Contextual)	3.47 (1.59)	−.49***	-
Convergent Validity Measures:
	3. Centrality (Importance; LGBIS)	3.98 (1.16)	.58***	−.54***	-
	4. Centrality (Importance; CSES)	3.99 (1.45)	.47***	−.45***	.60***	-
	5. Centrality (Importance/Salience; TFMSI)	4.42 (1.36)	.72***	−.68***	.80***	.65***	-
	6. Centrality (Importance/Salience; MMII)	4.91 (1.49)	.64***	−.58***	.80***	.62***	.83***	-
	7. Importance (MMII)	5.10 (1.61)	.53***	−.51***	.81***	.60***	.76***	.95***	-
	8. Salience (MMII)^a	4.43 (1.70)	.69***	−.53***	.58***	.49***	.76***	.80***	.58***	-
Predictive Validity Measure:
	9. Identity Concealment (LGBIS)	3.53 (1.49)	−.07*	.16***	−.29***	−.14***	−.19***	−.18***	−.23***	−.05	-
Discriminant Validity Measures:
	10. Depression (CES-D)	1.26 (0.68)	.11***	−.05	.07*	.10**	.11***	.06*	.02	.14***	.18***	-
	11. Anxiety (GAD-7)	1.13 (0.81)	.14***	−.05	.05	.09**	.10**	.07*	.02	.15***	.21***	.80***	-
	12. Stress (PSS)	2.00 (0.79)	.14***	−.05	.07*	.10**	.11**	.08**	.04	.15***	.20***	.84***	.79***	-
	13. Life Satisfaction (SWLS)	3.83 (1.50)	−.08**	.03	−.02	−.02	−.07*	−.01	.02	−.08**	−.20***	−.61***	−.49***	−.62***	-
	14. Self-Esteem (RSES)	2.66 (0.67)	−.06	.00	.04	−.03	−.02	.00	.05	−.07*	.26***	−.71***	−.60***	−.69***	.66***

Notes. *p ≤ .05, **p ≤ .01, ***p ≤ .001. ISQ = Identity Salience Questionnaire. LGBIS = Lesbian, Gay, and Bisexual Identity Scale. CSES = Collective Self-Esteem Scale. TFMSI = Three Factor Model of Social Identity. MMII = Multi-Group Measure of In-Group Identification. CES-D = Centre for Epidemiologic Studies Short Depression Scale. GAD-7 = Generalised Anxiety Disorder Questionnaire. PSS = Perceived Stress Scale. SWLS = Satisfaction with Life Scale. RSES = Rosenberg Self-Esteem Scale. ^aThis measure of identity salience includes the one salience item from the MMII, and an additional self-developed item of identity salience from Begeny and Huo (2017).

Convergent validity

As shown in Table 6, the chronic salience sub-scale showed moderate-to-strong positive correlations with identity centrality (rs = .47 to .72). Specifically, these correlations were the strongest among measures of identity centrality that included items referencing salience (rs = .64 to .72). A reverse pattern of relationships (i.e., negative correlations) with similar moderate-to-strong effects were found between the contextual salience sub-scale and all measures of identity centrality (rs = −.45 to −.68). These results provide support for convergent validity of the ISQ – higher levels of chronically thinking about an LGBTIQA+ identity (and less frequently thinking about the identity only when contextually relevant), is related to higher the levels of identity importance and frequency of thinking about an LGBTIQA+ identity (i.e., centrality-salience).

Predictive validity

The relationship between the ISQ sub-scales and identity concealment was mixed (Table 6). Whilst there was only a weak negative correlation between chronic salience and concealment, there was a small positive correlation between contextual salience and concealment. To explore this effect further we ran a forced-entry linear regression model whereby both dimensions of the ISQ were entered as predictors of identity concealment. The model was significant overall (F(2, 961) = 13.23, p < .001, R² = .03), and indicated that contextual salience was a significant positive predictor of identity concealment (b = 0.16, SE = 0.04, 95% CIs [0.09, 0.23], β = .17, p < .001), but that chronic identity salience did not predict identity concealment (b = 0.02, SE = 0.04, 95% CIs [−0.06, 0.09], β = .02, p = .674).

Discriminant validity

As also shown in Table 6, the chronic salience sub-scale was weakly correlated with greater levels of depression, anxiety, and stress symptoms, and less life satisfaction. There was no relationship between this sub-scale and self-esteem. Although some relationships were significant (likely due to the large sample size), we note that all correlations had weak effect sizes (rs = −.06 to .14). Further, the contextual salience sub-scale was not correlated with any mental health or well-being variables (rs = −.05 to .03). These findings provide support for the discriminant validity of the ISQ.

Discussion

The results from Study 2 provided support for a two-factor identity salience measure with adequate model fit and strong factor loadings. Within different LGBTIQA+ sub-groups, the factor structure of the ISQ remained stable. Further, correlation analyses provided support for the convergent, predictive, and discriminant validity of the ISQ. Although both the chronic and contextual sub-scales of the ISQ have shown very good internal consistency (Studies 1 & 2), the temporal (longitudinal) stability of these sub-scales is yet to be established.

Study 3

Study 3 aimed to assess the test-retest reliability of the ISQ (H8) within a sub-sample of LGBTIQA+ participants from Study 2.

Method

Participants

Participants from Study 2 who consented to being approached for an additional follow-up survey were included in Study 3. In total, 735 participants provided contact details at the completion of the survey and were invited to complete the follow up survey. In total, 487 participants started the survey, however a number of cases were excluded during data cleaning for the following reasons: 3 participants had duplicate responses, 6 did not consent to have their data included, 50 did not provide any responses after the demographic questionnaire, 6 participants were unable to have their data merged with their responses from the initial survey, and 104 participants endorsed a target identity that was different to their target identity within the first survey (thus, answering survey items with reference to a different identity).⁴ A final sample of 318 LGBTIQA+ participants provided follow-up data and were included. See Table 3 for a description of target identities.

Materials and procedure

Participants were contacted ~4 months after they completed the first survey either through e-mail contact (n = 216) or via the Prolific platform (n = 102). They were asked to complete an online survey with the same measures as described in Study 2. Only data for the ISQ measure are reported here.

Results and discussion

First, both sub-scales of the ISQ were examined for skewness at both timepoints, and all indicate normal distributions within this sub-sample. Further, both the chronic (α_T1 = .86; α_T2 = .83) and contextual (α_T1 = .90; α_T2 = .90) salience sub-scales had excellent estimates of internal consistency at both timepoints. Preliminary correlation analyses indicated strong positive relationships between Times 1 and 2 for both chronic and contextual salience (see Table 7). Further, a repeated-measures t-test revealed no significant differences in mean levels of chronic or contextual salience between timepoints. Finally, ICCs with a two-way mixed effect model, and absolute agreement, indicate that both the chronic and contextual salience have excellent test-retest reliability (i.e., ICC’s ≥.81; Table 7).

Table 7. Descriptive statistics, repeated-measures t-tests, zero-order correlations (r) for ISQ sub-scales across timepoints, and ICCs for test-retest reliability (study 3; N = 318).

	Time 1	Time 2	Repeated-Measures t-test	r(T1, T2) [95% CI]	ICC [95% CI]
	M (SD)	M (SD)
ISQ Chronic	3.18 (1.36)	3.18 (1.28)	t(315) = 0.00, p = 1.00, d = .00	.71 [.65, .76]	.83 [.79, .86]
ISQ Contextual	3.45 (1.64)	3.54 (1.51)	t(315) = −1.23, p = .221, d = −.07	.68 [.62, .73]	.81 [.76, .85]

Notes. All correlation coefficients and ICCs are significant at p < .001. T = Timepoint. ISQ = Identity Salience Questionnaire, ICC = Intraclass Correlation Coefficient.

The results from Study 3 provide strong evidence for the test-retest reliability of the ISQ in a sample of LGBTIQA+ participants. Thus far, the ISQ has a stable dual-dimensional structure (Study 1–2), with evidence of multiple forms of validity (Study 2) and reliability (Study 1–3: internal consistency; Study 3: temporal stability). As a final step in providing evidence for validity, we must assess how well the items themselves reflect the concepts of chronic and contextual salience (i.e., content validity).

Study 4

Study 4 aimed to explore the content validity of the ISQ items (H6) by asking social psychology researchers to examine how well the three items of each sub-scale reflect the constructs of chronic and contextual salience.

Method

Participants

Social psychology researchers, specifically those with familiarity or knowledge in areas relating to social identity and self-concept, were approached to complete this brief online survey by organization mail-lists and social media. Table 8 describes the recruitment sources of participants.⁵ A sample of 164 social psychology researchers commenced the survey exploring current understandings and conceptualizations of identity constructs. However, 57 participants did not provide responses to the second half of the survey in which the content validity of the ISQ was assessed, leaving a final sample of 107 researchers. Participants represented a variety of career stages, from PhD candidates to late-career academics (see Table 8). On average, participants indicated that they had at least some level of familiarity or knowledge in areas relating to social identity, identification, and/or self-concept (M = 5.68, SD = 1.15; on a scale from 1 [not at all familiar] to 7 [very familiar]).

Table 8. Descriptive statistics for participants’ career stage and recruitment source (study 4; N = 107).

	n	%
Recruitment Source:
Society of Australasian Social Psychologists (SASP)	39	36.4
Society for Personality and Social Psychology (SPSP)	17	15.9
Twitter	28	26.2
Facebook	6	5.6
E-mail from Colleagues/University Departments	13	12.2
Multiple Sources (e.g., SPSP and Twitter)	3	2.8
Unspecified	1	0.9
Career Stage:
PhD Candidate	32	29.9
Early-Career (up to 5 years from graduation)	30	28.0
Mid-Career (5–15 years from graduation)	24	22.4
Late-Career (15+ years from graduation)	17	15.9
Other (includes n = 2 Masters Students, n = 1 Retiree, and n = 1 Unspecified)	4	3.7

Measures and procedure

After consenting, participants responded to demographic questions, and were invited to complete the first part of the survey which instructed them to answer a series of open-ended questions regarding how they might conceptualize various constructs (not described in the current paper). Participants could then complete the second part of the survey which assessed the content validity of the ISQ.

Content validity measure

Following a similar design as developed by Lawshe (1975), participants were first presented with a broad operationalization of identity salience as we have defined it here (“The general frequency in which a particular (social) identity is thought of”), and then definitions of both chronic (“The extent to which the (social) identity is always/almost always frequently thought of”) and contextual (“The extent to which the (social) identity is only thought of when prompted by external or contextual factors”) salience. They were then instructed to indicate how useful or essential they thought each sub-scale item was in reflecting its respective conceptualization (chronic and contextual) by selecting one of three possible responses: 1 (item is not necessary), 2 (item is useful, but not essential), or 3 (item is essential).

Results and discussion

According to Lawshe (1975), assessing the content validity of items can be achieved by calculating a Content Validity Ratio (CVR) of the frequency in which items are deemed essential by panel experts, beyond the probability that items are deemed essential by chance. However, recent investigations of Lawshe’s (1975) work have claimed that the calculations involved in this CVR remain unclear (see Ayre & Scally, 2014; Wilson et al., 2012). Further, although Ayre and Scally (2014) and Wilson et al. (2012) provide some clarification on this, both indicate that CVR’s (using Lawshe’s (1975) method) can be appropriately determined for sample sizes up to 40 panel experts. Thus, given these barriers (i.e., with the current study having more than double the maximum sample size required for CVR computation), we opted to use a more liberal approach of looking at item-level frequencies to determine the content validity of the ISQ. Further, given previous insights in the mixed consensus on how identity constructs (e.g., salience, centrality) are operationalized and defined, and since we are asking experts to only rate the content of a small number of items, content validity in the current study was determined if the majority of experts rated ISQ items as being either useful or essential in reflecting their relevant constructs (i.e., items are rated as not content valid if they are deemed “not necessary” by the majority of content topic experts).

As demonstrated in Table 9, the vast majority of participants indicated that both the chronic salience (77.6–97.2%) and contextual salience (88.8–93.5%) items were either useful or essential reflections of their respective constructs. Thus, not only is the ISQ a valid measure of chronic and contextual salience in terms of its convergent, predictive, and discriminant associations with other constructs (Study 2), but also deemed valid regarding how well the items reflect the constructs of chronic and contextual salience, as defined within the current paper.

Table 9. Item-level frequencies of ISQ items being deemed useful or essential by social psychologists (study 4; N = 107).

	Item is not Necessary n (%)	Item is Useful, but not Essential n (%)	Item is Essential n (%)	Combined “Useful” or “Essential” n (%)
Chronic Salience Items:
Item 1: My [X] identity is often at the forefront of my mind	3 (2.8)	29 (27.1)	75 (70.1)	104 (97.2)
Item 2: Being [X] is mostly all that I think about	24 (22.4)	54 (50.5)	29 (27.1)	83 (77.6)
Item 3: I am often thinking about my [X] identity, despite what situation I am in	4 (3.7)	36 (33.6)	67 (62.6)	103 (96.3)
Contextual Salience Items:
Item 4: I only think about my [X] identity if someone or something has mentioned it	7 (6.5)	48 (44.9)	52 (48.6)	100 (93.5)
Item 5: My [X] identity is only at the forefront of my mind when someone brings it up	7 (6.5)	56 (52.3)	44 (41.1)	100 (93.5)
Item 6: I only think about my [X] identity when prompted	12 (11.2)	49 (45.8)	46 (43.0)	95 (88.8)

General discussion

Identity salience is a construct that has been defined, operationalized, and measured by researchers in various, conflicting, and sometimes problematic ways. Throughout this paper, we aimed to address these inconsistencies by (a) providing clarity on the conceptualization of identity salience though both its chronic and contextual elements of identity self-awareness, and (b) developing a measure of identity salience that accurately reflects this conceptualization. Specifically, we contend that there are two facets in the conceptualization of salience which represent the degree to which individuals think about their identity either perpetually (chronic salience) or only when contextually relevant (contextual salience). In accordance with this conceptualization, this new form of measuring identity salience (the ISQ) reliably and validly reflects these two facets (supporting hypotheses; Table 1) within LGBTIQA+ samples.

Chronic and contextual identity salience

Across Studies 1–2, we found a negative relationship between the facets of chronic and contextual salience. That is, higher degrees of chronically thinking about an identity is associated with lower degrees of only thinking about an identity when contextually prompted. Although our analyses indicate a linear relationship between these elements (visual assessment of scatterplots), we emphasize that these are both unique, but related, dimensions of the overall construct of identity salience. This was evidenced in two ways. First, we found structural validity of a two-factor model (Study 1–2), and also evidenced that this two-factor model had stronger fit in comparison to a one-factor model (Study 2). Second, differential relationships were found between each facet and the predictive validity variable of identity concealment. These latter findings broadly align with recent research that has conceptualized identity salience by its behavioral and contextual definitions (Brenner et al., 2023; Burke, 2023). Thus, while these constructs are (negatively) related, researchers should not assume that they are a unidimensional comparison of one another. For instance, when an individual scores lower on chronic salience, this need not only mean that they think about their identity only when contextually relevant (i.e., high context salience), as it could be argued that they simply do not think about their identity at all.

ISQ validation

Another main finding in this paper pertains to the relationships between identity salience and identity importance (or centrality). We found strong evidence that these constructs are related, with the magnitude of these relationships (especially for chronic salience) being stronger for measures of identity centrality that include items of chronic identity salience (e.g., Cameron, 2004; Leach et al., 2008). These results indicate that higher degrees of chronically thinking about an identity (and lower degrees of only thinking about an identity when contextually relevant) is associated with greater levels of the importance placed on the identity. This relationship is consistent with previous research that has conceptualized identity salience in the same way that we have here (e.g., Begeny & Huo, 2017; Quinn & Chaudoir, 2009; Quinn et al., 2014). Importantly, we note that the relationships between centrality (specifically when measured as identity importance) and salience are not strong enough to imply that they are concurrent constructs and highlight the need for their differential treatment in research (Brenner et al., 2014, 2023; Hinton et al., 2022; Thoits, 2020). Although this paper does not aim to examine the explicit differences between identity importance and salience, we note some preliminary findings on how these constructs differentially relate to other measures (e.g., identity concealment; Table 6).

In line with the earlier definitions and operationalizations of identity salience as a behavioral process, we predicted that identity concealment would relate more strongly to contextual than to chronic identity salience. We found a positive relationship between contextual salience and concealment, whilst there was no relationship between concealment and chronic salience after controlling for contextual salience (Table 6). This result provides an important contrast to former research on identity salience as operationalized by the degree of disclosure of the identity to other individuals (Brenner et al., 2023; Burke, 2023), such that non-disclosure of an identity (or concealment) is only related to the degree of thinking about the identity when that identity is prompted within one’s environment. This evidence also aligns with previous research among LGBTIQA+ groups (Hinton et al., 2022; Le Forestier et al., 2022), however we note that this relationship is also quite variable when examined within different LGBTIQA+ sub-groups. Although LGBTIQA+ sub-group analyses for these relationships were not a focal point in this paper, the results reported in supplementary materials (Tables S1–S6) indicate that (a) for chronic salience, the relationship with concealment is null except for those with a gender-diverse (non-binary or fluid) identity, and (b) for contextual salience, the relationship with concealment is null for those with a lesbian, bisexual, or other sexuality identity, but ranges from r = .18 to .32 amongst other LGBTIQA+ sub-groups. These differing relationships are likely due to differences between LGBTIQA+ sub-groups (see Hinton et al., 2022 for a discussion) rather than differences in scale structure (as evidenced in Study 2), but we caution over interpretation due to the likelihood that any comparisons would be underpowered. We encourage future researchers to further explore the relationships between identity salience and identity concealment, especially among different LGBTIQA+ sub-groups (e.g., see Salvati & Koc, 2022).

Finally, and partially consistent with previous research looking at the relationship between mental health, well-being, and LGBTIQA+ identity centrality (Begeny & Huo, 2017; Hinton et al., 2022; Utku & Sayılan, 2023), we find a weak positive relationship between symptoms of mental health and chronic identity salience (but no relationship with contextual identity salience, and virtually no relationships with well-being; Table 6). These results also provide insight on the differentiation of (chronic) identity salience and identity importance (or centrality), as virtually no relationships between mental health/well-being and identity centrality measures were found in the current study (Table 6), consistent with Hinton et al. (2022). Thus, we again encourage future researchers to explore how the constructs of identity importance and identity salience are differentially related to other psychosocial variables now that appropriate measurement of identity salience has been established.

Limitations and future directions

Although there are several notable strengths outlined in the current paper, including the comprehensive assessment of the dual-dimensional ISQ factor structure, evidence of its stability across identity groups (Study 2), temporal stability across timepoints (Study 3), and its content validity (Study 4), some limitations need to be noted. Indeed, whilst the evidence of content validity in Study 4 can be considered a strength of the current study, we also note that best practice guidelines suggest that item-level content validity ratings should be conducted prior to establishing evidence of factor structure (see Boateng et al., 2018). We made the decision to assess for content validity after the ISQ items had already been established and acknowledge that this is a deviation from standard test construction practices.

The studies presented in this paper have focused on the LGBTIQA+ community. While we believe it is important for this group to be examined in areas of identity research, the applicability of the ISQ to other identity groups would be an important avenue for future researchers to explore. We see no theoretical or methodological reasons why the factor structure and stability of the ISQ would differ in other identity groups. However, this might not be the case for the relationships between ISQ sub-scales and other outcomes. Researchers should continue exploring the relationships between ISQ factors and other measures, and validate the ISQ with social or role identity groups that differ in concealability, minority status, and permeability.

The ISQ has also been designed for global self-report purposes. That is, it may be limited in capturing how one thinks about their identity in chronic or contextual ways, from an ecological perspective. As an individual’s social environment is often fleeting, the degree to which the ISQ captures the momentary level of contextual identity salience may be limited. Indeed, most self-report measures capture a degree of recall bias, with the ISQ being no exception to this. Nonetheless, researchers should consider exploring the validation of chronic and contextual identity salience with more ecologically friendly research designs. Finally, the ISQ factors do not provide any insight into the valence of the targeted social identity. Unlike other constructs of social identification (e.g., identity solidarity or satisfaction; Leach et al., 2008) which frame their items through the understanding that an identity is a positive influence in an individual’s life, the ISQ offers a more neutral approach in just exploring the degree and frequency of thought. That is, the degree of identity self-awareness may be associated with both positive and negative affective experiences. Future research exploring the valence associations with the ISQ would be of considerable value.

Conclusion

Across four studies, we provide evidence of the stability, validity, and reliability of a new dual-dimensional scale of identity salience. We help bridge the gap in previous literature which has noted several inconsistencies in the operationalization and measurement of identity salience. Specifically, we provide evidence that identity salience consists of both the chronic degree in which an identity is thought of, and the context-dependent level of this thought process. We demonstrate that both salience facets are important aspects of the overall construct, they show stability in their structure across multiple identity groups, they are reliable (both internally and temporally), and they relate to similar constructs of social identification.

Adherence to ethical guidelines

All studies within this paper were approved by the Human Research Ethics Committee (HREC) at the Australian Catholic University, Melbourne.

Acknowledgments

We would like to acknowledge the GLOBE foundation for their partial contribution in assisting with participant recruitment funds, and the Australian Government Research Training Program Scholarship for supporting the first author of this submission.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Supplementary data

Supplemental data for this article can be accessed online at https://doi.org/10.1080/15298868.2024.2379841

Additional information

Funding

The author(s) reported there is no funding associated with the work featured in this article.

Notes

1. This survey was designed with the intent on answering several research questions (not described here), thus only the measures relevant to this study are reported.

2. The wording of items within this sub-scale reflected “sexual orientation” for participants with a sexuality target identity (see Table 3). For those with either a gender-diverse or intersex target identity, we changed the wording of the items to reflect their respective identities (e.g., “sexual orientation” was changed to either “gender-diversity” or “intersex identity”).

3. Although our value for RMSEA is considered sub-optimal by traditional cutoff values, we note here that researchers have no universal agreement of cutoff values for optimal model fit indices (Kline, 2016; Lai & Green, 2016). More consistently however, researchers do note that RMSEA values > .10 indicate “poor fit” (Kline, 2016). Whilst our RMSEA value of .08 is within this sub-optimal range (i.e., less than the cutoff for “poor fit,” but greater than the cutoff for “good fit”) and taking into account other model fit indices (all of which were in their acceptable range), we have left this model as reported. Further, on exploratory grounds, we assessed the modification indices of the model, which indicated that model fit improves if we allowed some item residuals to covary within the same factor. However, given the lack of theoretical reasoning to modify this model, we have kept the original fit indices as reported.

4. Of note, most participants (75%) had a target identity in the follow-up survey that was similar to their first target identity (e.g., a cisgender participant having a bisexual target identity in the first survey, and pansexual in the second survey). The remaining participants (25%) had incongruent target identities at both time points in that their target identity changed from a gender identity to a sexuality identity (or vice versa; e.g., a cisgender participant having a bisexual target identity in the first survey, but then indicating in the second survey that they are transgender, and thus having a targeted transgender identity). This aligns with recent research showing the fluid nature of these identity (e.g., Lyons et al., 2021).

5. The European Association of Social Psychology (EASP) was also contacted to advertise this online survey on their mail-list, but they promoted this survey on Twitter instead.

Appendix

The Identity Salience Questionnaire (ISQ)

Instructions: Please indicate your level of agreement for the following items using the response scale from 1 (Strongly Disagree) to 7 (Strongly Agree).

Table

		Strongly Disagree	Disagree	Somewhat Disagree	Neither Agree nor Disagree	Somewhat Agree	Agree	Strongly Agree
Chronic Salience Sub-Scale
	Item 1: My [X] identity is often at the forefront of my mind	1	2	3	4	5	6	7
	Item 2: Being [X] is mostly all that I think about	1	2	3	4	5	6	7
	Item 3: I am often thinking about my [X] identity, despite what situation I am in	1	2	3	4	5	6	7
Contextual Salience Sub-Scale
	Item 4: I only think about my [X] identity if someone or something has mentioned it	1	2	3	4	5	6	7
	Item 5: My [X] identity is only at the forefront of my mind when someone brings it up	1	2	3	4	5	6	7
	Item 6: I only think about my [X] identity when prompted	1	2	3	4	5	6	7

Notes: Items should be presented in a randomized order. [X] represents the target social identity of interest. Both sub-scales should be scored using the mean of all three items.