Examining the psychometric properties of the Athletic Coping Skills Inventory and Brief Cope in the firefighter population

Abstract

This study aimed to assess the psychometric properties of a modified sport psychology instrument in firefighters and explore the associations between psychological skills and coping strategies. Career firefighters completed an online survey assessing psychological skills and coping strategies. Confirmatory factor analyses (CFA) were performed on the modified Athletic Skills Coping Inventory (ACSI) and the Brief Cope (BC). Subsequent exploratory factor analysis (EFA) and covariance modeling were performed on the ACSI and bivariate correlations between the refined ACSI and BC were assessed. The refined ACSI included four factors and model fit exceeded fit recommendations. Nine significant relationships between the modified ACSI and BC were found. Excellent psychometric properties of the refined ACSI instrument provide preliminary evidence of the appropriateness, meaningfulness, and usefulness of the instrument in firefighters. However, modifications should be made that address the scale weaknesses before implementation.

Keywords:

• firefighters
• psychological skills
• coping
• structural equation modeling
• factor analysis

PUBLIC INTEREST STATEMENT

Firefighters are dispatched to unique and oftentimes dangerous emergencies. The high workload and variability of calls demand both physical and psychological prowess, similar to that of elite athletes. The stress inherent with the occupation, however, pose many health risks, thus elevating the prevalence of adverse physical and psychological health outcomes. Therefore, it is of the utmost importance to develop preventative programs to buffer the adverse outcomes. This article explores the possibility of using a sport psychological skills instrument in the fire population. Additionally, it assesses the relationship between psychological skills and coping profiles in a sample of career firefighters. Our hope is that a psychological skill assessment tool can be created that uniquely captures skills specific to the firefighter population.

1. Introduction

In 2017, the National Fire Protection Association (NFPA) estimated there were 1 million firefighters (35% career and 65% volunteer) in the United States who responded to 34 million calls; 1.3 million fire incidents and 22.3 million emergency medical aid calls (Evarts & Stein, 2019). Although there has been a decline in fire incidents since the 1980s, the rate of medical aid calls have dramatically risen (Evarts & Stein, 2019). The high workload and variability of calls demand both physical and psychological prowess (Coffey et al., 2016; Fischer et al., 2017; Holmér & Gavhed, 2007). Essential physical job tasks involve ladder raise and hose lay, operation of machinery for cutting and creating openings in structures, extraction and extrication, search and rescue operations, and providing first aid and cardiac care (Rodriguez, 2001). In addition, the International Association of Firefighters (IAFF) outlines a number of cognitive and psychosocial necessities including the ability to perform under chaotic conditions, maintain focus, communicate, adapt, recall and execute orders, work and cooperate with a close-knit team, and self-assess and report any diminishing physical or psychological capacities affecting job performance (Rodriguez, 2001).

The physical and psychological demands, along with the specialized training, of firefighters are similar to that of elite athletes preparing for a sport performance (Gnacinski, Ebersole et al., 2015; Gnacinski, Meyer et al., 2015; Norwood & Newman, 2014). Firefighters, like elite athletes, typically complete rigorous physical (e.g., cardiovascular and weight training) and skill-based (e.g., throw ladders, lay hose, ropes, rescue tools) training regimes, along with situation-based preparations (e.g., search and rescue, vehicle extrication, forcible entry) to maximize individual efforts. The physical and mental preparation of firefighters has led to their classification as tactical athletes (Norwood & Newman, 2014).

While on duty, firefighters are dispatched to a diverse number of calls including motor vehicle accidents, hazardous material spills, structure fires, or medical aid (Evarts & Stein, 2019). When firefighters arrive to emergency calls, they work under pressure in situations that may rapidly change and evolve and are exposed to potentially traumatic events involving scared individuals, gruesome accidents, and deaths. The physical and psychological stress inherent with the occupation poses many health risks, which elevate the prevalence of adverse physical (Yip et al., 2016) and psychological health outcomes (Chiu et al., 2011). For example, firefighters are at an increased risk for physical injury (Evarts & Molis, 2018), sleep disturbances (Carey et al., 2011; L. J. Smith et al., 2019), psychological distress (Dean et al., 2014), and depression (Carey et al., 2011; Fullerton et al., 2004). Further, firefighters are at an elevated risk for alcohol and binge drinking behavior (Haddock et al., 2012; Tomaka et al., 2017). The potential outcome of this is increased risk of post-traumatic stress symptomology and disorder (PTSD) (Alghamdi et al., 2017; Haslam & Mallon, 2003; Nydegger et al., 2011), suicide (Martin et al., 2017; Stanley et al., 2016; Tiesman et al., 2015), and mortality (Fahy & Molis, 2018; Kunadharaju et al., 2011).

A limited number of protective factors have been identified that reduce psychological distress, namely use of various coping strategies (Carey et al., 2011; Tomaka et al., 2017), higher levels of social support (Cook & Mitchell, 2013; Prati & Pietrantoni, 2010), and self-efficacy (Regehr et al., 2003). Researchers have found firefighters’ use of instrumental and emotional support (Carpenter et al., 2015; Chamberlin & Green, 2010), humor (Jahnke et al., 2016; Sliter et al., 2014) and high self-efficacy (Lambert et al., 2012) can potentially buffer the development of PTSD and stress. Given the high rate of adverse health outcomes for this occupation, preventative research and programming with this sub-category of tactical athletes is warranted.

Although physical fitness programs are common in the fire industry, few departments offer specialized training aimed at improving and enhancing cognitive, psychosocial, and psychological well-being (Rodriguez, 2001). Researchers have found that various psychological interventions including mindfulness (Kaplan et al., 2017), cognitive behavioral therapy (Difede et al., 2007), critical incident stress debriefing (Hokanson & Wirth, 2000), and relaxation techniques (Mitani et al., 2006) are beneficial for the fire population. However, research on these techniques (e.g., critical incident stress debriefing, cognitive behavioral therapy, etc.) were used to reduce rather than prevent symptoms. The development of psychological skills could be used to fill this void. The use of psychological skills to enhance individual sport performance (Andrew et al., 2007; Gould et al., 1981; Maddison & Prapavessis, 2005; Spieler et al., 2007) served as a natural starting point for the use of this training with tactical athletes because preliminary results indicate firefighters use psychological skills while performing on the job (Gnacinski, Meyer et al., 2015). Despite the similarities between sport performers and tactical athletes, few research studies on firefighters have been conducted to specifically investigate the use of these skills to enhance performance (Lauria et al., 2016; Lorello et al., 2016) or as a preventative strategy to combat adverse mental health outcomes.

One of the necessary steps in this process is to develop or identify instruments to assess the impact of psychological skills training on performance. Previous research in the sport and military population have utilized instruments that assess individual psychological constructs such as resilience (Kaplan et al., 2017; Ogińska-Bulik & Kobylarczyk, 2015), self-efficacy (Lambert et al., 2012), or motivation (Gnacinski et al., 2015). Although these instruments provide beneficial information about a single psychological construct, multifaceted instruments allowing researchers to assess multiple psychological skills (Gnacinski, Meyer et al., 2015), are warranted. One potential instrument is the Athletic Coping Skills Inventory-28 (ASCI) (R. E. Smith et al., 1995) because it is designed to measure seven specific psychological factors related to competitive performance in athletic populations (Andrew et al., 2007; Young & Knight, 2014). The ACSI-28 has been reported to demonstrate sufficient internal reliability with α ranging from.62—.86 (Andrew et al., 2007; Omar-Fauzee et al., 2014; Ozcan & Gunay, 2017; R. E. Smith et al., 1995) and test re-test reliability with α between subscales ranging from .62—.87 (Andrew et al., 2007; Ozcan & Gunay, 2017).

Firefighters, like elite athletes, face unknown situations, must be in peak physical condition, and have ample mental preparation (Gnacinski et al., 2015). The ACSI is proposed to measure factors such as goal setting skills, the ability to cope with adversity, and high concentration and motivation, which may be particularly important for this population (Marquardt et al., 2019). The ACSI, however, has not been previously assessed in firefighters and the items originally proposed in the instrument may not be phrased (e.g., competition vs. job) in a manner that fits the firefighter population or job setting. Thus, to properly assess the effectiveness of a psychological skills training program for firefighters, there is a need to develop a measurement tool that is both psychometrically sound and suitable for the specific population with which it will be used (Dillman, 2014; Kline, 2015).

Therefore, the primary purpose of this study was to assess the psychometric properties of a modified sport psychology instrument (ACSI) for firefighters. If model fit recommendations for the instruments were not met, we would re-examine the factor structure of the modified sport psychology instrument using exploratory factor analysis (EFA) and covariance modeling. A secondary purpose of the study was to complete an exploratory examination of the relationship between the refined psychological skills instrument and coping strategies utilized by firefighters.

2. Methods

The research project was approved by the University Institutional Review board. Participants were informed on response anonymity, the volunteer nature of the survey, and study withdrawal process prior to beginning the survey, and then provided informed consent for participation. A random sample of fire chiefs (n = 45) located primarily in the Pacific Northwest were sent a personalized email asking if they were interested in having their firefighters complete a survey exploring psychological skill and coping strategies. Email addresses for the fire chiefs were obtained via a search of district fire websites. A total of 10 fire chiefs (22% response rate) indicated interest in participating in the study; each of those 10 fire chiefs were emailed the survey link to distribute to the firefighters in their department. In addition, a snowball method of recruitment was used in that the fire chiefs were encouraged to send the link to other departments they felt would be interested in the study.

2.1. Instrumentation

The survey, which included two instruments (modified-Athletic Coping Skills Inventory and Brief COPE) and a participant questionnaire, was developed using Qualtrics Survey Software (Qualtrics Inc., Provo, UT). The survey packet was sent to two veteran firefighters (one active, one retired) to ensure face and content validity.

2.1.1. Modified Athletic Coping Skills Inventory

The Athletic Coping Skills Inventory-28 (ACSI) has been proposed as a valid and reliable instrument (R. E. Smith et al., 1995) to assess psychological skills in a number of athletic populations including football (Spieler et al., 2007), baseball (Kimbrough et al., 2007), rugby (Andrew et al., 2007; Yeomans et al., 2019), equestrian (Salma & Meyers, 2019) and a group of diverse Olympic champion athletes (Gould et al., 2002). The ACSI is a 28-item instrument designed to measure seven factors, with each factor containing four items. Subscales include peaking under pressure (PEAK), freedom from worry (FREE), coping with adversity (COPE), concentration (CONC), goal setting and mental preparation (GOAL), confidence and achievement motivation (CONF), and coachability (COACH). The ACSI was not originally designed for firefighters and need to be modified to assess multidimensional psychological skills of firefighters. All 28 original items of the ACSI were retained; however, each item was retooled (i.e., modified language) to fit the firefighter population (e.g., “While competing, I worry about … ” was retooled to “While on the job, I worry about … ”). To maintain the integrity of the original concepts, the amount of retooling was kept to a minimum. Participants answered each item using a five-point Likert scale (1 = Never, 2 = Rarely, 3 = Sometimes, 4 = Often, 5 = Always).

2.1.2. Brief Cope

To assess coping strategies, the Brief Cope (BC; Carver, 1997) with a modified Likert-scale was used. The BC is a 28-item instrument, designed to measure 14 factors, with two items in each. The subscales include: acceptance (ACPT), humor (HUM), religion (REL), planning (PLAN), active coping (AC), positive reframing (POSR), denial (DENY), self-blame (BLM), behavioral disengagement (BDIS), substance use (SUB), active self-distraction (ASD), instrumental support (INS), venting (VENT), and emotional support (EMOS). All 28 original items were retained, and individuals were instructed to answer items based on what they typically do when under a lot of stress. In order to reduce complexity and increase ease of answering the questionnaire, participants used a five-point Likert scale (1 = Never, 2 = Rarely, 3 = Sometimes, 4 = Often, 5 = Always).

2.1.3. Participant questionnaire

Participants provided demographic information (e.g., sex, age, gender, etc.), background information (e.g., years worked in the fire industry, level of care, geographic location, etc.), and information regarding perceived health, stress on-duty, stress off-duty, and sleeping patterns. Continuous variables are reported as (mean±SD) and categorical variables are reported as (n; percentage).

2.2. Data analysis

Data was exported from Qualtrics and analyses were conducted in Statistical Package for Social Sciences Version 25.0 (SPSS, Inc., Chicago, IL). Missing responses for each survey item were calculated for each respondent; those missing responses to more than 10% of the items in the modified-ACSI (i.e., 3 or more missing responses) or BC (i.e., 4 or more missing responses) were excluded from the analysis. The remaining missing data were replaced with the rounded mean score (Kline, 2015; Tabachnick et al., 2001) of the respective item for analysis purposes.

Normality of data was assessed using histograms and skewness and kurtosis values. Univariate outliers were assessed using z-scores with a cut-off value of |3.3| and possible multivariate outliers for each scale were assessed using Malahanobis distance. The cut-off value was identified using a chi-square table with degrees of freedom and p-value of .01 (Kline, 2015). Following cleaning procedures, data were analyzed using CFA procedures for the modified ACSI and BC. Because model fit did not meet recommended guidelines as outlined in the literature (Bryant & Yarnold, 1995; Kline, 2015) for the ACSI, an EFA was conducted to identify a more parsimonious solution. The final EFA solution was then tested using a more rigorous covariance model approach (Kline, 2015). Bivariate correlations using structural equation modeling was done on the total summed score of the refined ACSI and each subscale of the BC.

2.2.1. Confirmatory factor analysis modified Athletic Coping Skill Inventory and Brief Cope

The full sample was used to conduct a CFA in Analysis of Moment Structures (AMOS) Version 25 software (IBM Corp., Armonk, NY) on the modified ACSI and the BC. Model fit indices were evaluated based on a priori values to evaluate the originally proposed factor structures. The relative goodness-of-fit indices computed were the Comparative Fit Index (CFI; ≥ .95), Tucker-Lewis Index (TLI; ≥ .95), Root Mean Square Error of Approximation (RMSEA ≤ .06), and Bollen’s Incremental Fit Index (IFI; ≥ .95) (Bryant & Yarnold, 1995; Hu & Bentler, 1999; Kline, 2015). The likelihood ratio statistic (Chi-square or CMIN) was also assessed, but because it is heavily influenced by sample size, it was not used as the primary assessment of model fit (Brown, 2014; Kline, 2015). Because model fit criteria were not met for the ACSI, EFA and covariance modeling procedures were conducted to assess for a more psychometrically sound factor structure.

2.2.2. Identification of a refined Athletic Coping Skill Inventory

The full sample was re-analyzed using maximum likelihood extraction EFA. Bartlett’s test for sphericity (<.001) and Kaiser-Meyer Olkin Measure of Sampling Adequacy (≥.70) for sampling adequacy were both assessed for violations (Leech et al., 2014). Items with loadings less than .40 were removed as well as items with cross-loadings at .30 or greater (Leech et al., 2014). Items were removed one at a time. Cronbach’s alpha estimate of internal consistency (set a priori at ≥ .70 and ≤ .89) and bivariate correlations between items and Cronbach’s alpha estimate of internal consistency were also used to make removal decisions (Leech et al., 2014).

2.2.3. Validation analysis of the refined Athletic Coping Skill Inventory

The refined ACSI scale identified during the EFA process was then re-assessed using covariance modeling. The same criteria utilized for the initial CFA was used to assess model fit (Brown, 2014; Kline, 2015). If needed, modifications were made to the model and re-analyzed with the same criteria. Finally, a composite score bivariate correlational analysis was conducted to assess the percentage of variance in responses on the refined version of the scale accounted for in the original ACSI (Ware et al., 1996).

2.2.4. Bivariate correlation analysis

The full sample was used to assess the relationship between the refined ACSI and each of the subscales in the BC. All items from the refined ACSI were summed to create a “Total ACSI” score for each individual. Using structural equation modeling, the “Total ACSI” manifest variable was correlated to each latent variable subscale of the BC.

3. Results

3.1. Data cleaning

A total of 358 full-time firefighters completed the survey; however, 14 (4.0%) were excluded due to missing data. Fifty-one (14.0%) cases were identified as outliers (25 univariate and 26 multivariate) and were also removed. Thus, responses from 293 participants were included for analysis.

3.2. Sample characteristics

Participant demographic and perceived health information (n = 293) is provided in Table 1. The age range of participants was 21 to 67 years (41.6 ± 10.67 years) and the years worked in the fire industry ranged from 6 months to 47.5 years (15.7 ± 9.98 years). The majority of firefighters were male (n = 278; 94.9%), Caucasian (n = 266; 90.8%), and predominately worked in a city (n = 158; 53.9%).

Table 1. Demographic and Perceived Health Information

Characteristics	N	%	Characteristics	N	%
Sex			Health
Male	278	94.9	Excellent	116	39.6
Female	9	3.1	Good	147
Prefer not to answer	3	1.0	Average	28	50.2
Unknown	3	1.0	Unknown	2	9.6.7
Education			Stress On-Duty
High school or GED	66	22.5	High	6	2.0
Technical college	28	9.6	Moderately high	74	25.3
Associate degree	72	24.6	Moderate	124	42.3
Bachelor’s degree	111	37.9	Moderately low	72	24.6
Master’s degree	11	3.8	Low	15	5.1
Doctoral degree	3	1.0	Unknown	2	0.7
Married			Stress Off-Duty
Single, never married	27	9.2	High	5	1.7
Married/domestic partner	240	81.9	Moderately high	31	10.6
Divorced	18	6.1	Moderate	85	29.0
Separated	3	1.0	Moderately low	113	38.6
Prefer not to answer	2	0.6	Low	57	19.5
Widowed	1	0.3	Unknown	2	0.7
Ethnicity			Sleeping pattern
Caucasian	266	90.8	Very good, regular	31	10.6
African American	1	0.3	Satisfactory	135	46.1
Hispanic	9	3.1	Disturbed/Restless	95	32.4
Asian/Pacific Islander	7	2.4	Difficulty falling asleep	26	8.9
Native American	7	2.4	Don’t sleep	4	1.4
Prefer not to answer	4	1.4	Unknown	2	0.7
Other	7	2.4
Location			Level of care
Small town (1,001–20,000)	26	8.9	EMT-Basic	143	48.8
Large town (20,001–100,000)	64	21.8	EMT-Advanced	37	12.6
City (100,001–300,000)	158	53.9	EMT-Paramedic	110	37.5
Large city (over 300,000)	43	14.7

3.3. Confirmatory factor analysis Brief Cope

The CFA of the fourteen-factor, 28-item BC indicated good model fit to the sample data. All of the fit indices calculated exceeded recommended values (χ² [259] = 364.184; CFI = .969; TLI = .955; IFI = .970; RMSEA = .037; Figure 1) and all item-factor loadings were statistically significant (p ≤ .001) and ranged from .43 to .97. Significant correlations between first-order latent variables ranged from .15 to .87 (Table 2).

Table 2. Correlations Between First-Order Latent Variables of the Brief Cope

	AC	PLAN	POSR	ACPT	REL	EMOS	DENY	ASD	SUB	BDIS	BLM	VENT	INS	HUM
AC	1.0	.87***	.31***	.60***	.09	.01	−.28***	.02	−.14	−.53***	−.10	.09	.14	.02
PLAN		1.0	.44***	.22	.12	.19*	−.004	.16*	−.15*	−.28**	−.15*	.18*	.28***	−.11
POSR			1.0	.30**	.32***	.38***	.03	.28***	−.20**	−.41***	−.15*	.19*	.38***	.04
ACPT				1.0	.07	−.09	−.46***	.21	.10	−.08	.03	−.28**	−.21***	.15
REL					1.0	.52***	.02	.06	−.22***	−.11	.09	.29***	.22***	−.11
EMOS						1.0	.02	.25**	−.09	−.18*	.05	.74***	.71***	.03
DENY							1.0	.11	.18**	.50***	.22**	−.05	.07	.10
ASD								1.0	.15*	.08	.05	.08	−.01	.19*
SUB									1.0	.45***	.17**	−.04	−.14*	.15*
BDIS										1.0	.50***	−.16	−.32***	.12
BLM											1.0	.07	.004	.22**
VENT												1.0	.55***	.17*
INS													1.0	−.09
HUM														1.0

AC = active coping; PLAN = planning; ACPT = acceptance; REL = religion; EMOS = Emotional support; DENY = denial; ASD = active self-distracting; SUB = substance use; BDIS = behavioral distraction; BLM = blame; VENT = vent; INS = instrumental support; HUM = humor.

*** ≤.001, ** ≤.01, * ≤.05.

Figure 1. Confirmatory Factor Analysis First-Order Brief Cope

Chi Sq = Chi Square (χ2); CFI = Comparative Fit Index; TLI = Tucker-Lewis Index; IFI = Bollen’s Incremental Fit Index; RMSEA = Root Mean Square Error of Approximation, df = degrees of freedom, p = alpha level.

3.4. Confirmatory factor analysis modified Athletic Coping Skills Inventory

The CFA of the modified ACSI indicated poor fit to the sample data. The goodness-of-fit indices did not meet recommended values (CFI = .848; TLI = .825; IFI = .850; RMSEA = .067; Figure 2) and modification indices suggested a number of meaningful cross-loadings. Moreover, all correlations between first-order latent variables (e.g., CONF, PEAK, FREE, etc.) except GOAL to FREE were statistically significant and ranged from .15 to .85 (Table 3).

Table 3. Correlations between First-Order Latent Variables of the Athletic Coping Skills Inventory

	GOAL	CONF	COACH	CONC	COPE	PEAK	FREE
GOAL	1.0	.67***	.28***	.30***	.39***	.27***	−.065^#
CONF		1.0	.56***	.85***	.79***	.53***	.34***
COACH			1.0	.59***	.82***	.39***	.43***
CONC				1.0	.77***	.55***	.45***
COPE					1.0	.56***	.34***
PEAK						1.0	.15*
FREE							1.0

*** ≤.001, ** ≤.01, * ≤.05, ^# >.05

Figure 2. Confirmatory Factor Analysis First-Order Modified ACSI

Chi Sq = Chi Square (χ2); CFI = Comparative Fit Index; TLI = Tucker-Lewis Index; IFI = Bollen’s Incremental Fit Index; RMSEA = Root Mean Square Error of Approximation, df = degrees of freedom, p = alpha level.

3.5. Identification of a refined Athletic Coping Skills Inventory

Initial EFA of the modified ACSI using the full sample extracted seven dimensions. Fifteen items were eliminated due to low loadings or high cross-loadings, and two items were eliminated due to high correlation values. The resulting four-factor, 11-item scale consisted of items from FREE (three items), PEAK (three items), GOAL (three items), COACH (two items). Cronbach’s alpha was within an acceptable range for all factors: FREE (α = .796), GOAL (α = .749), PEAK (α = .755), and COACH (α = .826). The eleven items accounted for 72.16% of the variance with all item loadings ≥ .57 (Table 4). Data factorability of the final solution was met with KMO = .702 and a significant Bartlett’s test of sphericity.

Table 4. Pattern Matrix for Exploratory Factor Analysis—Refined ACSI

Component	Item	Loading	Loading	Loading	Loading
Freedom From Worry	While on the job, I worry about making mistakes or failing to come through.*	.970
	I think about and imagine what will happen if I fail or screw up.*	.656
	I worry quite a bit about what others think about my job performance.*	.616
Goal Setting	I tend to do lots of planning about how to reach my goals.		.871
	On a daily or weekly basis, I set very specific goals for myself that guide what I do.		.687
	I set my own performance goals on the job.		.573
Peaking Under Pressure	I tend to perform on the job under pressure because I think more clearly.			.781
	I make fewer mistakes when the pressure’s on because I concentrate better.			.753
	The more pressure there is during a situation, the more I enjoy it.			.639
Coachability	When someone criticizes me, I become upset rather than helped. *				−.874
	When someone tells me how to correct a mistake I’ve made, I tend to take it personally and feel upset. *				−.780
Eigenvalue		3.06	2.17	1.66	1.05
Cronbach’s Alpha		.796	.749	.755	.826

*Items reversed scored

3.6. Validation analysis of the refined Athletic Coping Skills Inventory

Covariance modeling of the refined ACSI using the full sample indicated good model fit to the sample data. All of the fit indices calculated exceeded recommended values (χ² [38] = 44.199, p = .226; CFI = .994; TLI = .991; IFI = .994; RMSEA = .024; Figure 3). All factor loadings were significant (p ≤ .001) and ranged from .61 to .87. Significant correlations between latent variables ranged from .18 to .48 (Table 5). Scores for the refined 11-item ACSI were moderately correlated (r = .67, R² = .45) with the scores for the 28-item ACSI.

Table 5. Correlations between First-Order Latent Variables of the Refined Athletic Coping Skills Inventory

	GOAL	COACH	PEAK	FREE
GOAL	1.0	.19**	.18*	−.02^#
COACH		1.0	.25***	.48***
PEAK			1.0	.19**
FREE				1.0

*** ≤.001, ** ≤.01, * ≤.05, ^# >.05

Figure 3. Covariance Model of Refined ACSI

Chi Square = Chi Square (χ2); CFI = Comparative Fit Index; TLI = Tucker-Lewis Index; IFI = Bollen’s Incremental Fit Index; RMSEA = Root Mean Square Error of Approximation, df = degrees of freedom, p = alpha level.

3.7. Bivariate correlation analysis

Bivariate correlations between the total ACSI score and the subscales of the BC are presented in Table 6 and Figure 4. Firefighter total ACSI scores were significantly correlated with Brief Cope factors active coping (R² = .46), planning (R² = .33), positive reframing (R² = .28), acceptance (R² = .29), instrumental support (R² = .15), active self-distraction (R² = .14), blame (R² = −.35), behavioral disengagement (R² = −.40), and denial (R² = −.21).

Table 6. Bivariate correlations between Brief Cope subscales and total refined Athletic Coping Skills Inventory scores

	Total ACSI
Active Coping	.46***
Planning	.33***
Positive Reframing	.28***
Acceptance	.29**
Religion	−.08^#
Emotional Support	.08^#
Humor	.00^#
Instrumental Support	.15*
Venting	.01^#
Blame	−.35***
Behavioral Disengagement	−.40***
Substance Use	.01^#
Active Self-Distraction	.14*
Denial	−.21***

*** ≤.001, ** ≤.01, * ≤.05, ^# >.05

Figure 4. Bivariate correlation analysis between composite ACSI scores and the BC subscales

Chi Sq = Chi Square (χ2); CFI = Comparative Fit Index; TLI = Tucker-Lewis Index; IFI = Bollen’s Incremental Fit Index; RMSEA = Root Mean Square Error of Approximation, df = degrees of freedom, p = alpha level. Items with *** ≤.001, ** ≤.01, * ≤.05.

4. Discussion

Firefighters are dispatched to unique and oftentimes dangerous emergencies. The high workload and variability of calls demand physical and psychological proficiency (Coffey et al., 2016; Fischer et al., 2017), similar to that of elite athletes (Gnacinski, Meyer et al., 2015; Norwood & Newman, 2014). The stress inherent with the occupation pose many health risks which may elevate the prevalence of adverse physical and psychological health outcomes (Chiu et al., 2011; Yip et al., 2016). Therefore, it is of the utmost importance to develop preventative programs to buffer adverse outcomes. Given the similarities between firefighters and elite athletes, it is possible to develop a psychological skill training program to increase psychological skills and buffer poor outcomes. Thus, the use of a sport psychology tool for the firefighter population may be warranted. Therefore, the purpose of this paper was to explore the possibility of using a sport psychological skills instrument (i.e., ACSI) in the fire population and assess the relationship between psychological skills and coping profiles.

The major finding of our study was that the initial retooled ACSI had poor model fit in the firefighter population. The EFA and covariance modeling led to the removal of items until a refined ACSI, containing eleven of the original twenty-eight items, was accepted. The refined ACSI exceeded model fit recommendations; however, it only included four of the original seven factors. A secondary purpose of this study was to assess the relationship between total psychological skills (refined ACSI) and coping strategies using the BC. The BC met model fit recommendations; therefore, no modifications were made to the instrument. Bivariate correlations between the refined ACSI and the BC were assessed. There were six significant positive relationships between adaptive coping strategies and ACSI, and three significant negative relationships between problematic coping strategies and ACSI.

4.1. Refined Athletic Coping Skills Inventory

An 11-item, four factor solution emerged from our analysis. The refined scale included items from the “coachability,” “goal setting and mental preparation,” “peaking under pressure,” and “freedom from worry” factors, but no items from the original “confidence and achievement motivation,” “concentration,” or “coping with adversity” factors were retained. Of the four factors retained, three factors contained three of the original four items and one factor contained two of the original four items. The refined 11-item scale was then subjected to a covariance model analysis. Statistically, the refined scale met a priori guidelines for model fit (Kline, 2015), which offers preliminary evidence of a psychometrically-sound psychological skills instrument for firefighters. However, use of the refined ACSI instrument for targeted interventions is cautioned until psychometric issues are addressed, more rigorous testing and statistical assessment is completed, and further testing (e.g., invariance testing) is done to assess potential instrument weaknesses.

A number of weaknesses within the instrument were identified in this process and should be addressed prior to adoption. Specifically, several methodological flaws in item format, factor item valence switching, and item applicability to the target audience were identified when considering recommendations for survey design (Decoster, 2005; Dillman, 2014; Tait & Voepel-Lewis, 2015; Van Sonderen et al., 2013). The ACSI included items that were double-barred (combining two questions in one; e.g., “While on the job, I worry about making mistakes or failing to come through” and “It is easy for me to keep distracting thoughts from interfering with something I am watching or listening to”) or nonmonotonic (asking a question that could be answered for two different reasons; e.g., “When someone tells me how to correct a mistake I’ve made, I tend to take it personally and feel upset” and “I make fewer mistakes when the pressure’s on because I concentrate better”). Double-barreled and nonmonotonic items cause ambiguity and make it difficult for the participant to respond (Decoster, 2005; Dillman, 2014). Similarly, surveys should avoid including items that are both positively and negatively worded (valence switching) as this too can cause participant confusion and inaccuracy (Dillman, 2014; Roszkowski & Soven, 2010; Van Sonderen et al., 2013). Future research should attempt to rewrite items for each factor with valences in the same direction and restructure the double-barreled and nonmonotonic items.

Finally, we must also consider the purpose of the scale and the need to measure all of the psychological constructs of interest proposed in the ACSI. The original ACSI scale contained seven factors, while our analysis only retained four. The total scores on the 11-item scale and the original 28-item ACSI were only moderately correlated (r = .67). The correlation value is lower than recommendations for determining if the modified scaled measured an acceptable proportion of the variance (r = .90; Ware et al., 1996) in responses on the original scale. The correlation value may indicate the two scales are not capturing the same phenomenon, with the modified scale providing reduced insight due to items and factors that were not retained.

The factors that were eliminated (i.e., CONF, COPE and CONC), however, contained items that may not be relevant or worded in an effective manner for firefighters, particularly in regards to the situations firefighters face during job performance (Avraham et al., 2014; Boyle et al., 2010; Garner et al., 2016). For example, the items written to assess COPE for the competitive athlete population focused on how an athlete is able to respond to setbacks and maintain a positive attitude (e.g., “When things are going badly, I tell myself to keep calm and this works for me” and “I remain positive and enthusiastic while on the job, no matter how badly things are going”) and items written to assess CONF focus more on the athletes confidence in skills and consistently giving maximum effort (e.g., “I don’t have to be pushed to practice or train hard—I give 100%”). Similarily, the items written to assess CONC for the competitive athletes focuses on how an athlete is able to block out distractions (e.g., “It is easy for me to keep distracting thoughts from interfering with something I am watching or listening to”).

Although firefighters do have to cope with adversity, concentrate, and feel confident in their skill set, we believe the items in those factors did not reflect the occupation duties associated with firefighting and led to removal. The inability to assess potentially valuable constructs supports further refinement to the modified version of the scale to develop a comprehensive instrument for use in the firefighter population. Therefore, to produce a more parsimonious and psychometrically sound instrument, our data suggest revising items that can adequately assess a firefighter’s psychological skills, particularly in the CONF, CONC, and COPE domains.

4.2. Psychological skills and coping strategies

The BC demonstrated good model fit; thus, modifications to the instrument were not necessary (Kline, 2015). Bivariate correlations between the fourteen coping strategies and the refined total ACSI score from all items in the refined instrument were calculated. The coping strategies defined as planning, active coping, positive reframing, instrumental support, active self-distraction, acceptance, denial, behavioral disengagement, and blame were all related to total ACSI score. It is interesting to note that the coping strategies typically described as adaptive and efficient (i.e., active coping, planning, positive reframing, active self-distract, instrumental support, acceptance) were positively related to psychological skills while the generally problematic strategies (e.g., denial, behavioral disengagement, blame) were negatively related.

4.3. Limitations and future research

Although our study identified a psychometrically sound instrument, our data cleaning analysis used conservative a priori standards and therefore, a large portion of outliers were identified and removed from the sample. While previous researchers have indicated that transformation of the data leads to marginal improvements (Tabachnick et al., 2001), it is possible that the removal of those individuals influenced the results of our study. Additionally, a true cross-validation analysis was not performed to assess the refined ACSI, and therefore, our findings should be interpreted with caution. Further, although we contacted fire chiefs from many geographic locations with differing call volumes, due to the survey anonymity, it was difficult to assess to what extent the sample adequately represented the firefighter population. Future research using a cross-validation sample should be done to allow for a true CFA to be conducted on the refined scale. Additionally, invariance testing in a new sample was not conducted and should be done in order to fully assess potential group difference (e.g., firefighters working in a city v rural area, firefighters with more experience v rookies, etc.) and allow for more scale precision. Lastly, our analysis revealed a relationship between higher psychological skills and adaptive coping profiles, therefore, future research should explore the possibility that psychological skills training may positively influence individual coping flexibility (Heffer et al., 2017). For this population in particular, developing a preventative program to counter or buffer the high adverse health outcomes can be very beneficial.

5. Conclusion

Overall, good psychometric properties of the refined ACSI instrument provide compelling preliminary evidence of the appropriateness, meaningfulness, and usefulness of the instrument in firefighters. Our results support the assessment of psychological skills and coping mechanisms in the firefighter population and provide insight into an overall adaptive coping mechanism profile for firefighters who score higher on psychological skills use. However, we recommend further modifications be made to develop a more parsimonious and psychometrically sound instrument that will result in a more clinically relevant tool to best assess these constructs in firefighters. Specifically, more items should be written for each psychological skill factor (e.g., concentration, coping with adversity, etc.) and refinement to existing items should be done to provide participants more clarity when responding.

Additional information

Funding

The authors received no direct funding for this research.

Notes on contributors

Russell T. Baker

The research team key activities involve psychometric analysis of various psychosocial instruments related to individual well-being and injury. Recently, we have performed psychometric analysis on quality of life and disablement instruments in participants who are physically active or suffering from musculoskeletal injury. Psychometrically sound instruments that are valid for their intended populations are important for researchers and clinicians to better understand patient conditions and determine the effects of physical and psychological interventions.